A Radar to Watch You in Your Car

Used to be, when we said the walls had ears, it meant there were microphones hidden in them. Now, when we say the walls have eyes, will it mean they have radar?

Maybe so, at least in your car. Vayyar Imaging, a firm based in Tel Aviv, says it has a radar chip that can form a three-dimensional view of what’s going on inside a car as well as outside of it. Right now, though, it’s concentrating on the inside, because there’s a regulatory push to having in-cabin observation in place in the early 2020s, and Vayyar thinks it has a head start.

Everyone else seems to be banking on cameras, including infrared cameras, to do this job. You can buy such a system right now: the Cadillac CT6 Super Cruise. That car can drive itself for extended periods, but it uses a camera to scrutinize the driver for signs of distraction or fatigue to make sure that, if a problem comes up that the system can’t solve, it can safely hand control back to the human.

Using radar inside the car is novel—even weird.

The first question is how Vayyar does it. It starts by putting most things on a chip that sits on a board inside a thin package that could be hidden in the roof or dashboard. There are 48 radars on the chip, 40 of which are hooked to antennas on a board. (The others are held in reserve, for self-monitoring purposes.) 

The 40 beams are projected simultaneously throughout the car’s interior, and their reflections are also received simultaneously. This format is known as mimo, for multiple beams in and multiple beams out.  

“We generate thousands of beams by combination,” says Vayyar CEO Raviv Melamed, speaking from Tel Aviv, which is where he and his colleagues co-founded the company. “We transmit beams number 1, 2, and 3, and so forth, in parallel, and we receive them on each of the 40 receivers, and the combination gives us virtual points—many, many signals. The points look like a voxel—a volumetric cell, the kind you’d get from lidar. We analyze them all in parallel, so it’s very fast.”

This video shows a number of such point clouds. In one, a man is sitting in the front passenger’s seat and leaning far forward. Next to him is a radar point cloud suggesting the bended form. The car’s system can read that cloud to infer that releasing an airbag right now might break the guy’s neck. Better to hold back that release until the crash has begun, when the guy may find himself in a bag-safe zone.

The second question is—why use radar instead of cameras? The answer, Melamed says, is fourfold. It will (he promises) cost less than the several thousand dollars you need for a high-performance stereo camera; it can take in the entire cabin, not just focus on a particular seat; it can see through things that would block a camera’s view; and it doesn’t give details of people’s faces, and so can’t invade their privacy.

Radar can penetrate seats, people, and a blanket thrown over a baby that might be sleeping in the car seat. It can even detect the baby’s breathing. Not leaving a baby behind by mistake is one of the things future regulators are going to insist upon. 

Of course, radar can’t penetrate everything. I asked Melamed how the system handles beams reflected off the metal sides. He admits that it isn’t easy, but says Vayyar has developed algorithms to “nullify” those reflections. “Today, we are the best at that,” he asserts. 

But the metal problem is why the company doesn’t attempt to use the in-cabin chip to sense the environment outside the car. To do that, another chip, exactly like the one used inside the car, could be placed on the door or at the front. Thus placed, it could look as far as 100 meters away, producing a point cloud that would compete against lidar. True, lidar would still be the more accurate sensor, but Vayyar’s radar would still beat it in bad weather. But that outside-the-car project’s on the back burner, for now.

The company has forged partnerships with the auto suppliers Brose, based in Coburg, Germany; Valeo, based in Paris, and Faurecia, based near Paris. 

It’s just one more example of how radar is striking back at sexier sensors. Proponents of radar say it’s unfair to pit new sensors against existing radar sets that were never really optimized for use in cars. There’s plenty of room for improvement there, they claim. A number of startups are chasing it.

One company, called Lunewave, builds a spherical radar antenna that can transmit and receive beams from all directions. It’s based on ideas that date back to the 1940s but were never economical enough or compact enough for automotive applications until Lunewave began using 3D printing to craft its intricately structured sphere.

Researchers at Fraunhofer Institute for Reliability and Microintegration, in Berlin, are working on a combined camera-radar system called KameRad. The system processes the data in the sensor itself so that it can send a command, such as “stop now,” straight to the car, bypassing other processors. That could shave life-saving milliseconds off the response time.

From the beginning, some designers of self-driving cars have preferred radar and cameras to lidars. Elon Musk is the most famous lidar-basher, but even mighty Mercedes-Benz chose radar as the fundamental sensor for Bertha, its original self-driving prototype. 

Old technologies don’t necessarily die. And if they’re constantly changing, they’re not even old.