Phantom Auto engineer Ben Shukman watches the passing cars carefully before pulling out of the MGM Grand parking lot and onto busy Tropicana Avenue in Las Vegas. Shukman skillfully merges into fast-flowing traffic as we chat about how the uncharacteristic rain here this week has led to an uptick in accidents over the last few days.
But Shukman is not sitting next to me in the driver’s seat of Phantom’s Lincoln MKZ, and he hasn’t felt a drop of rain in weeks. Shukman is remotely controlling the car from Mountain View, Calif., more than 500 miles away.
Gif: Phantom/IEEE Spectrum
In the first such demonstration on public roads, Phantom Auto hopes to convince skeptical carmakers and wary regulators that the best backup for today’s experimental autonomous vehicles (AVs) is nothing more or less than an old-fashioned human driver.
“An autonomous vehicle company might have a system that works 95 or even 99 percent of the time, but that last 1 percent is a very difficult piece of the puzzle to solve,” says Phantom CEO Shai Magzimof. “We’re here to do that hardest part.”
Magzimof is sitting in the Lincoln’s driver’s seat for safety but does not touch the steering wheel throughout the entire 15 minute drive, which includes edging around pedestrians in a gas station and reversing in the MGM parking lot.
But that simplicity is deceptive, says Magzimof. What has many in the autonomous vehicles industry convinced that teleoperated cars are impractical is the issue of latency—the inevitable delays that data packets experience as they wend their way from one end of a wireless connection to the other, and back again.
“At the very beginning, the car would hiccup every second,” admits Magzimof. “Imagine FaceTime on your phone in the passenger seat.” A few frozen frames of video are acceptable when you’re chatting with your parents, but the same could prove fatal while driving at highway speeds. A dead spot in the wireless network could shut the car down completely.
Phantom tackles this problem from multiple angles. For a start, its system uses multiple wireless networks simultaneously. Our ride used AT&T and Verizon, but Phantom says it can also loop in T-Mobile and Sprint if needed. Before Phantom will allow a car to be remotely controlled, the company maps the strength of each network signal over the car’s service (or geofenced) area.
“All we need is our system connected to power, and then you can put it in an Uber to drive around the city,” says Magzimof. “In some places AT&T works better, in others it’s Verizon. It’s very rare to find a place with no connection at all, and those we would know in advance.”
Phantom has mapped Palo Alto, Mountain View, and parts of Las Vegas so far, and intends to map San Francisco in the coming weeks. (This may be linked to its first customer, which Phantom says it will announce soon. Several companies are currently testing automated vehicles in San Francisco, notably GM/Cruise and Zoox.)
Phantom also sends data redundantly, with each video frame going through different channels, and sends the most critical data—to control brakes, for example—across all available networks at the same time. It has also developed a machine-learning algorithm that jumps between networks to avoid the dynamic throttling that networks often use to prevent heavy users from hogging all the bandwidth.
“We’ve done enough work to know when it’s about to happen, and we just don’t let it happen. We jump off the network first. It’s a pretty cool orchestration with thousands of parameters,” says Magzimof.
Even so, network glitches and slowdowns are inevitable, and Phantom’s system is designed to degrade gracefully. If the bandwidth suddenly drops, the left and right screens will pixelate first. If all the high-speed LTE and 4G networks go down, Magzimof says Phantom can keep working on sluggish 3G, albeit with only low-quality video, steering, and brakes. If the worst should happen and the connection drops entirely, Phantom has developed a basic AI assist system that slows the vehicle to a stop and turns on emergency flashers.
Phantom Auto engineer Ben Shukman operates a vehicle from more than 500 miles away.Photo: Phantom
During our test ride, a dash-mounted screen showing Shukman in Mountain View (as well as an audio link to chat with him) made it apparent that the system was experiencing little latency. Shukman’s driving felt very safe, if on the cautious side. One particular scenario, as Shukman smoothly exited the gas station and crossed a turn lane, felt like something an autonomous vehicle would struggle to replicate.
“Humans can do this delicate social dance in a way computers can’t,” notes Magzimof. Phantom initially envisages its system as a backup for those edge cases in which AI fails: gravel roads, accidents like the recent situation where a motorbike hit a Cruise AV [PDF] in San Francisco, or even picking up and dropping off ride-share passengers. “Imagine a human greeting you getting into your car, or you being able to tell them exactly where you want to be dropped off,” he says.
Phantom currently has five control systems set up in Mountain View. At each system, one human could oversee perhaps five automated vehicles. Training on the system takes about a week, starting with simulated driving, then teleoperation on a closed course before the driver passes a test to drive a car on public roads.
While the regulatory side of vehicle teleoperation is still largely unexplored, lawyer and Phantom cofounder Elliot Katz sees smooth roads ahead. “Many regulators around the world don’t know how to swallow this at first, but when they see it, they trust it more than the computer,” he says. “California has characterized us as an ADAS (Advanced Driver Assistance System) for AVs.”
If that holds elsewhere, Phantom should be able to deploy with the licenses or testing permits that some states and countries currently require. And they do plan to deploy very soon, with cars at first, and trucks in the future—and especially as wireless networks switch to 5G.
“We’re looking at a market that could be hundreds of vehicles next year and after that, thousands,” says Magzimof. “Our best customers, the ones who instantly want to engage this, are those who are closest to deploying their own AVs. They’re 98 percent of the way there and want us to close that safety gap with something that they, and their riders, can trust.”
Mark Harris is an investigative science and technology reporter based in Seattle, with a particular interest in robotics, transportation, green technologies, and medical devices. He’s on Twitter at @meharris and email at mark(at)meharris(dot)com. Email or DM for Signal number for sensitive/encrypted messaging.