Night Vision for Self-Driving Cars

An image of the far infrared camera called Viper that the startup AdaSky has developed for self-driving cars.
Image: AdaSky

Elon Musk famously thinks that cars can be made to drive themselves without relying on expensive laser-ranging lidars. But while Tesla is moving ahead with one fewer sensor than most self-driving car companies, a new startup wants them to add yet another—an infrared camera.

AdaSky is developing a far infrared thermal camera called Viper that it says can expand the conditions that automated cars will be able to operate in, and improve safety.

“Today’s sensors are not good enough for fully self-driving cars and that’s where we come in,” says Dror Meiri, vice president of business development at AdaSky. “We think infrared (IR) technology can bridge the gap from Level 3 all the way to Levels 4 and 5.”

Level 3 vehicles need a human driver ready to take control at a moment’s notice. Passengers in Level 4 and 5 self-driving cars can read a book or go to sleep. For that to happen, a car’s sensors must be able to provide a detailed and dependable 3D image of the car’s surroundings and other road users.

However, existing sensors all have their weak points. High-resolution lidars struggle in rain, fog, and snow. Radars can punch through bad weather but deliver less detailed information, while cameras suffer the same limitations as human eyes when faced with bright sunlight, glare, or nighttime conditions.

Passive infrared vision can help fill in these gaps. It spots differences in the heat emitted by objects in the road ahead. Warm-blooded humans and animals are naturally prominent, while road surfaces stand out from nearby vegetation. Oncoming headlights, direct sunlight, and abrupt lighting changes (which drivers experience when exiting a tunnel) do not wash out the entire scene, as they can for normal cameras.

Far infrared technology has been around for decades in night vision goggles and security cameras. But its automotive applications have been limited to date. One only company, Autoliv, currently offers an infrared camera for cars, which is sold as a pricey (US $2,000+) option on a few luxury models from Audi, BMV, Cadillac, Mercedes-Benz, and Rolls Royce

That camera is fairly low resolution, 0.08 megapixels, and needs to recalibrate itself briefly every few minutes, during which time it does not operate. “That’s not good enough for fully autonomous cars,” says Meiri. “Our camera is higher resolution [0.3 megapixels] and can operate continuously at 60 frames per second.”

AdaSky has also developed its own computer vision to interpret the infrared video, which can identify pedestrians, cyclists, and animals a long way beyond the range of a car’s headlights. Infrared video also contains different artifacts and presents separate vision problems than optical video. AdaSky believes there is additional commercial advantage to having the only system working at higher resolutions.

If the company can succeed in making an affordable and reliable infrared camera, it’s likely to find favor with self-driving car makers. “The knock on thermal IR systems has always been cost,” says Karl Iagnemma, CEO of nuTonomy. “It’s only a complementary sensor, but could fill a nice role in a fully built-out sensor configuration.”

Tarek El-Gaaly, senior research scientist at Voyage, agrees: “Infrared sensors provide some visual information, as well as the 2D shape of objects, and are very useful for nighttime where vision via optical cameras fail. There is no one-sensor-does-it-all for autonomous vehicles, it really comes down to fusing multiple sensors together into a sensor suite to complement each other.”

AdaSky says its system is currently being evaluated by several car companies and suppliers, and the startup hopes to start mass production in 2020 or 2021.

Advertisement

Cars That Think

IEEE Spectrum’s blog about the sensors, software, and systems that are making cars smarter, more entertaining, and ultimately, autonomous.
Contact us:  p.ross@ieee.org

Editor
Philip E. Ross
New York City
Contributor
Willie D. Jones
New York City
 
Senior Writer
Evan Ackerman
Washington, D.C.
Contributor
Mark Harris
Seattle
 

Newsletter Sign Up

Sign up for the Cars That Think newsletter and get biweekly updates, all delivered directly to your inbox.

Advertisement