The August 2022 issue of IEEE Spectrum is here!

Close bar

New Pedestrian Detector from Google Could Make Self-Driving Cars Cheaper

A deep learning system works 60 times faster than previous methods

3 min read
New Pedestrian Detector from Google Could Make Self-Driving Cars Cheaper
Image: Google

Google’s self-driving cars roam the sunny streets of Mountain View, Calif., in public but much of the technology that powers them has never seen the light of day. Yesterday, attendees at the IEEE International Conference on Robotics and Automation (ICRA) in Seattle got a rare glimpse into a new safety feature the tech giant is working on.

Anelia Angelova, a research scientist at Google working on computer vision and machine learning, presented a new pedestrian detection system that works on video images alone. Recognizing, tracking, and avoiding human beings is a critical capability in any driverless car, and Google’s vehicles areduly festooned with lidar, radar, and cameras to ensure that they identify people within hundreds of meters.

But that battery of sensors is expensive; in particular, the spinning lidar unit on the roof can cost nearly $10,000 (or more if for multiple units). If autonomous vehicles could reliably locate humans using cheap cameras alone, it would lower their cost and, hopefully, usher in an era ofrobotic crash-free motoring all the sooner. But video cameras have their issues. “Visual information gives you a wider view [than radars] but is slower to process,” Angelova told IEEE Spectrum.

At least it used to be. The best video analysis systems use deep neural networks—machine learning algorithms that can be trained to classify images (and other kinds of data) extremely accurately. Deep neural networks rely on multiple processing layers between the input and output layers. For image recognition, the input layer learns features of the pixels of an image. The next layer learns combinations of those features, and so on through the intermediate layers, with more sophisticated correlations gradually emerging. The output layer makes a guess about what the system is looking at.

Modern deep networks can outperform humans in tasks such as recognizing faces, with accuracy rates of over 99.5 percent. But traditional deep networks applied to pedestrian detection are very slow, dividing each street image into 100,000 or more tiny patches, explains Angelova, and then analyzing each in turn. This can take seconds or even minutes per frame, making them useless for navigating city streets. Long before a car using such a network has identified a pedestrian, it might have run the person over.

imgThese example images show Google’s deep learning system detecting pedestrians in different situations. The system performed 60 times faster than previous methods.Image: Anelia Angelova/Google

Angelova’s new, high speed pedestrian detector has three separate stages. The first is a deep network, but one that slices up the image into a grid of just a few dozen patches rather than tens of thousands. This network is trained to do multiple detections simultaneously at multiple locations, picking out what it thinks are pedestrians. The second stage is another network that refines that result, and the third is a traditional deep network to deliver the final word on whether the car is seeing a person or, say, a mailbox.

However, because that slow, accurate network only analyzes a small portion of the image where pedestrians are likely to be, the whole process runs much faster—between 60 and 100 times quicker than the best previous networks, says Angelova. Running on graphics processors similar to those in Google’s self-driving cars and fed street images, the system was trained in about a day. It could then accurately identify pedestrians in around 0.25 seconds. (The researchers use a well-known pedestrian image database, rather than video from Google cars, because it lets them compare their results to previous networks).

“That’s still not the 0.07 seconds needed for real time use,” admits Angelova. Self-driving cars need to know almost instantly whether they are facing pedestrians or not, in order to safely take evasive action. “But it means [the new system] could be complementary in case other sensors fail,” she says.

As more powerful processors become available and the capacity of the neural network increases, Angelova expects that performance will improve. “For networks with even larger fields of view, one can consider even more speedups,” she says. By the time self-driving cars are available for the general public to buy, their distinctive spinning lidars may have disappeared altogether. 

The Conversation (0)

Self-Driving Cars Work Better With Smart Roads

Intelligent infrastructure makes autonomous driving safer and less expensive

9 min read
A photograph shows a single car headed toward the viewer on the rightmost lane of a three-lane road that is bounded by grassy parkways, one side of which is planted with trees. In the foreground a black vertical pole is topped by a crossbeam bearing various instruments. 

This test unit, in a suburb of Shanghai, detects and tracks traffic merging from a side road onto a major road, using a camera, a lidar, a radar, a communication unit, and a computer.

Shaoshan Liu

Enormous efforts have been made in the past two decades to create a car that can use sensors and artificial intelligence to model its environment and plot a safe driving path. Yet even today the technology works well only in areas like campuses, which have limited roads to map and minimal traffic to master. It still can’t manage busy, unfamiliar, or unpredictable roads. For now, at least, there is only so much sensory power and intelligence that can go into a car.

To solve this problem, we must turn it around: We must put more of the smarts into the infrastructure—we must make the road smart.

Keep Reading ↓Show less