Google’s self-driving cars roam the sunny streets of Mountain View, Calif., in public but much of the technology that powers them has never seen the light of day. Yesterday, attendees at the IEEE International Conference on Robotics and Automation (ICRA) in Seattle got a rare glimpse into a new safety feature the tech giant is working on.

Anelia Angelova, a research scientist at Google working on computer vision and machine learning, presented a new pedestrian detection system that works on video images alone. Recognizing, tracking, and avoiding human beings is a critical capability in any driverless car, and Google’s vehicles areduly festooned with lidar, radar, and cameras to ensure that they identify people within hundreds of meters.

But that battery of sensors is expensive; in particular, the spinning lidar unit on the roof can cost nearly $10,000 (or more if for multiple units). If autonomous vehicles could reliably locate humans using cheap cameras alone, it would lower their cost and, hopefully, usher in an era ofrobotic crash-free motoring all the sooner. But video cameras have their issues. “Visual information gives you a wider view [than radars] but is slower to process,” Angelova told IEEE Spectrum.

At least it used to be. The best video analysis systems use deep neural networks—machine learning algorithms that can be trained to classify images (and other kinds of data) extremely accurately. Deep neural networks rely on multiple processing layers between the input and output layers. For image recognition, the input layer learns features of the pixels of an image. The next layer learns combinations of those features, and so on through the intermediate layers, with more sophisticated correlations gradually emerging. The output layer makes a guess about what the system is looking at.

Modern deep networks can outperform humans in tasks such as recognizing faces, with accuracy rates of over 99.5 percent. But traditional deep networks applied to pedestrian detection are very slow, dividing each street image into 100,000 or more tiny patches, explains Angelova, and then analyzing each in turn. This can take seconds or even minutes per frame, making them useless for navigating city streets. Long before a car using such a network has identified a pedestrian, it might have run the person over.

imgThese example images show Google’s deep learning system detecting pedestrians in different situations. The system performed 60 times faster than previous methods.Image: Anelia Angelova/Google

Angelova’s new, high speed pedestrian detector has three separate stages. The first is a deep network, but one that slices up the image into a grid of just a few dozen patches rather than tens of thousands. This network is trained to do multiple detections simultaneously at multiple locations, picking out what it thinks are pedestrians. The second stage is another network that refines that result, and the third is a traditional deep network to deliver the final word on whether the car is seeing a person or, say, a mailbox.

However, because that slow, accurate network only analyzes a small portion of the image where pedestrians are likely to be, the whole process runs much faster—between 60 and 100 times quicker than the best previous networks, says Angelova. Running on graphics processors similar to those in Google’s self-driving cars and fed street images, the system was trained in about a day. It could then accurately identify pedestrians in around 0.25 seconds. (The researchers use a well-known pedestrian image database, rather than video from Google cars, because it lets them compare their results to previous networks).

“That’s still not the 0.07 seconds needed for real time use,” admits Angelova. Self-driving cars need to know almost instantly whether they are facing pedestrians or not, in order to safely take evasive action. “But it means [the new system] could be complementary in case other sensors fail,” she says.

As more powerful processors become available and the capacity of the neural network increases, Angelova expects that performance will improve. “For networks with even larger fields of view, one can consider even more speedups,” she says. By the time self-driving cars are available for the general public to buy, their distinctive spinning lidars may have disappeared altogether. 

The Conversation (0)

We Need More Than Just Electric Vehicles

To decarbonize road transport we need to complement EVs with bikes, rail, city planning, and alternative energy

11 min read
A worker works on the frame of a car on an assembly line.

China has more EVs than any other country—but it also gets most of its electricity from coal.

VCG/Getty Images
Green

EVs have finally come of age. The total cost of purchasing and driving one—the cost of ownership—has fallen nearly to parity with a typical gasoline-fueled car. Scientists and engineers have extended the range of EVs by cramming ever more energy into their batteries, and vehicle-charging networks have expanded in many countries. In the United States, for example, there are more than 49,000 public charging stations, and it is now possible to drive an EV from New York to California using public charging networks.

With all this, consumers and policymakers alike are hopeful that society will soon greatly reduce its carbon emissions by replacing today’s cars with electric vehicles. Indeed, adopting electric vehicles will go a long way in helping to improve environmental outcomes. But EVs come with important weaknesses, and so people shouldn’t count on them alone to do the job, even for the transportation sector.

Keep Reading ↓Show less