The December 2022 issue of IEEE Spectrum is here!

Close bar

A Radar to Watch You in Your Car

Vayyar says there are at least four good reasons to monitor passengers with radar instead of cameras

4 min read
Gif showing the Vayyar Imaging radar point cloud on the left, of the man at the steering wheel seen at the right.
Gif: Vayyar Imaging/IEEE Spectrum

Used to be, when we said the walls had ears, it meant there were microphones hidden in them. Now, when we say the walls have eyes, will it mean they have radar?

Maybe so, at least in your car. Vayyar Imaging, a firm based in Tel Aviv, says it has a radar chip that can form a three-dimensional view of what’s going on inside a car as well as outside of it. Right now, though, it’s concentrating on the inside, because there’s a regulatory push to having in-cabin observation in place in the early 2020s, and Vayyar thinks it has a head start.

Everyone else seems to be banking on cameras, including infrared cameras, to do this job. You can buy such a system right now: the Cadillac CT6 Super Cruise. That car can drive itself for extended periods, but it uses a camera to scrutinize the driver for signs of distraction or fatigue to make sure that, if a problem comes up that the system can’t solve, it can safely hand control back to the human.

Using radar inside the car is novel—even weird.

The first question is how Vayyar does it. It starts by putting most things on a chip that sits on a board inside a thin package that could be hidden in the roof or dashboard. There are 48 radars on the chip, 40 of which are hooked to antennas on a board. (The others are held in reserve, for self-monitoring purposes.) 

The 40 beams are projected simultaneously throughout the car’s interior, and their reflections are also received simultaneously. This format is known as mimo, for multiple beams in and multiple beams out.  

“We generate thousands of beams by combination,” says Vayyar CEO Raviv Melamed, speaking from Tel Aviv, which is where he and his colleagues co-founded the company. “We transmit beams number 1, 2, and 3, and so forth, in parallel, and we receive them on each of the 40 receivers, and the combination gives us virtual points—many, many signals. The points look like a voxel—a volumetric cell, the kind you’d get from lidar. We analyze them all in parallel, so it’s very fast.”


This video shows a number of such point clouds. In one, a man is sitting in the front passenger’s seat and leaning far forward. Next to him is a radar point cloud suggesting the bended form. The car’s system can read that cloud to infer that releasing an airbag right now might break the guy’s neck. Better to hold back that release until the crash has begun, when the guy may find himself in a bag-safe zone.

The second question is—why use radar instead of cameras? The answer, Melamed says, is fourfold. It will (he promises) cost less than the several thousand dollars you need for a high-performance stereo camera; it can take in the entire cabin, not just focus on a particular seat; it can see through things that would block a camera’s view; and it doesn’t give details of people’s faces, and so can’t invade their privacy.

Radar can penetrate seats, people, and a blanket thrown over a baby that might be sleeping in the car seat. It can even detect the baby’s breathing. Not leaving a baby behind by mistake is one of the things future regulators are going to insist upon. 

Of course, radar can’t penetrate everything. I asked Melamed how the system handles beams reflected off the metal sides. He admits that it isn’t easy, but says Vayyar has developed algorithms to “nullify” those reflections. “Today, we are the best at that,” he asserts. 

But the metal problem is why the company doesn’t attempt to use the in-cabin chip to sense the environment outside the car. To do that, another chip, exactly like the one used inside the car, could be placed on the door or at the front. Thus placed, it could look as far as 100 meters away, producing a point cloud that would compete against lidar. True, lidar would still be the more accurate sensor, but Vayyar’s radar would still beat it in bad weather. But that outside-the-car project’s on the back burner, for now.

The company has forged partnerships with the auto suppliers Brose, based in Coburg, Germany; Valeo, based in Paris, and Faurecia, based near Paris. 

It’s just one more example of how radar is striking back at sexier sensors. Proponents of radar say it’s unfair to pit new sensors against existing radar sets that were never really optimized for use in cars. There’s plenty of room for improvement there, they claim. A number of startups are chasing it.

One company, called Lunewave, builds a spherical radar antenna that can transmit and receive beams from all directions. It’s based on ideas that date back to the 1940s but were never economical enough or compact enough for automotive applications until Lunewave began using 3D printing to craft its intricately structured sphere.

Researchers at Fraunhofer Institute for Reliability and Microintegration, in Berlin, are working on a combined camera-radar system called KameRad. The system processes the data in the sensor itself so that it can send a command, such as “stop now,” straight to the car, bypassing other processors. That could shave life-saving milliseconds off the response time.

From the beginning, some designers of self-driving cars have preferred radar and cameras to lidars. Elon Musk is the most famous lidar-basher, but even mighty Mercedes-Benz chose radar as the fundamental sensor for Bertha, its original self-driving prototype

Old technologies don’t necessarily die. And if they’re constantly changing, they’re not even old.

The Conversation (0)

We Need More Than Just Electric Vehicles

To decarbonize road transport we need to complement EVs with bikes, rail, city planning, and alternative energy

11 min read
A worker works on the frame of a car on an assembly line.

China has more EVs than any other country—but it also gets most of its electricity from coal.

VCG/Getty Images
Green

EVs have finally come of age. The total cost of purchasing and driving one—the cost of ownership—has fallen nearly to parity with a typical gasoline-fueled car. Scientists and engineers have extended the range of EVs by cramming ever more energy into their batteries, and vehicle-charging networks have expanded in many countries. In the United States, for example, there are more than 49,000 public charging stations, and it is now possible to drive an EV from New York to California using public charging networks.

With all this, consumers and policymakers alike are hopeful that society will soon greatly reduce its carbon emissions by replacing today’s cars with electric vehicles. Indeed, adopting electric vehicles will go a long way in helping to improve environmental outcomes. But EVs come with important weaknesses, and so people shouldn’t count on them alone to do the job, even for the transportation sector.

Keep Reading ↓Show less