The main problem of SSD networks is the minimal area of recognition.
It simple to train SSD network to predict big objects that just a few on a image, but much harder to predict many small object which is a big problem if we are talking about self-driven cars, cause the most cameras has lenses where object in few meters already small enough to cause problems for SSD networks.

That was one of the problems which i encountered during making Mask-SSD network for one of my projects.