Cars That Think iconCars That Think

Schematic of Zoox's proposed bidirectional, all-electric, self-driving car

Secretive Robotaxi Startup Zoox Prepares for Real-World Testing

There was not much to see when I arrived at the entrance to the small business park in the sleepy Silicon Valley town of San Carlos. Signage for three of its units were blank but, upon closer inspection, I could just make out the word Zoox on a door. I had tracked the company to this location using visa and property records, and was hoping to get a glimpse of what it was up to.

The name Zoox probably does not mean much to its neighbors, or even to electric vehicle fans seeking the birthplace of Tesla Motors, which started in this very building in 2004. But behind these glass walls and shuttered garages is one of the most buttoned-up and most valuable autonomous vehicle startups in the world.

first investigated this stealthy start-up for IEEE Spectrum in 2015, revealing its ambition to develop a fully-automated robotaxi from the ground up. Two years later, Zoox is valued at $1.55 billion, employs over 200 people, and has more than 50 PhDs and post-docs in computer vision, artificial intelligence, and automotive design—many previously employed at Tesla. It remains one of the lowest profile companies in the Valley, shunning press coverage and sharing nothing more than a stylized infinity logo on its website.

My latest investigation has uncovered patents filed by the company, progress on the development of its technology, and testing details. Zoox’s vision—the same one that propelled it when it was a small startup working out of a decommissioned fire station on the grounds of Stanford University—remains on track. Its aim: to transform transportation by offering a Level 5, or completely driverless, mobility service.

“When I’m on that campus, there’s an app in my phone, and I can press a button and one of our autonomous vehicles… will drive through the road network to come to rest where I am,” said Tim Kentley-Klay, founder and CEO of Zoox, at a meeting at Stanford last year. “When I’m inside, I push another button and that vehicle takes me to where I want to go… And when I get out, that vehicle then goes and does that for someone else... That entire dance happens without human intervention.”

When I visited Stanford in early 2015, Zoox was working with an innovative research ‘mule’ initially devised by KTH Royal Institute of Technology, in Stockholm. The vehicle was outfitted with lidar, radar, video, and ultrasonic sensors. The car had an electric motor in each wheel, and was designed to be bidirectional, meaning it could move equally well forwards and backwards. Several researchers at KTH subsequently left to join Zoox. Sources familiar with the company say that although Zoox is still testing using KTH vehicles, there has been considerable evolution in the Swedish university’s design.

Zoox experimented with internal combustion and fuel cell technologies but, according to sources, it has returned to the concept of an all-electric drivetrain for its first prototype. The project has already reached the “body in white” stage, where the shape of the car has been largely finalized, in preparation for the production of the chassis and bodywork.

Recently issued patents (filed in late 2015) provide more detail about what Zoox has been working on. The paperwork includes images that match early design sketches of the car. One patent suggests the car may use novel materials, such as a foam insulation frame with a matrix of triangular, circular, and hexagonal shapes that provide structural support and serve as conduits for cables, hoses and wiring.

The outside of the vehicle may have a flexible fabric skin that could carry advertising or the name of a transportation provider. But the most radical change is in the vehicle’s interior. There is no driver’s seat or controls. Instead, the car’s four seats face towards the middle of the vehicle. Passengers awaiting collection might be alerted by the car “winking” its headlights; the vehicle might confirm that it has the right person via facial recognition.

A patent concerning teleoperation seems to call into doubt Kentley-Klay’s claim that Zoox vehicles can operate with “zero human intervention.” A source familiar with the company confirms that each Zoox vehicle can be remotely operated if it suffers an accident or encounters an unfamiliar situation. One remote operator would have responsibility for multiple vehicles. The patent provides an example of a teleoperator user interface showing sensor data, remaining charge, and local speed limits.

Zoox is also fairly unique among autonomous vehicle startups in considering the safety of other road users. One of the patents mentions AI algorithms that can estimate the probability that nearby human drivers, cyclists, or pedestrians are behaving irrationally, and adjust the car’s trajectory accordingly. When the Zoox vehicle predicts that someone might stray into its path, it would direct flashing lights and a steerable beam of sound at them. If an accident appears unavoidable, external bladders would rapidly fill with air to cushion the impact.

Zoox will obviously need self-driving software to control its new vehicle. The company was launched with a 10-year, royalty-free, exclusive license to Stanford University’s self-driving car software, developed in part by Zoox’s co-founder, Jesse Levinson. In early 2016, Zoox applied for an autonomous vehicle testing permit from the California Department of Motor Vehicles (DMV). Its testing regime started with a single Lexus SUV. A fleet of five modified Toyota Highlanders—operating as Level 3 autonomous vehicles with a safety driver ready to take over at any time—now form the core of its testing fleet.

Testing the company’s fully driverless development car could prove trickier. While Zoox’s base at Stanford is on private property, the company’s road testing agreement with the university (obtained by IEEE Spectrum under a Freedom of Information Act request) stipulates that Zoox researchers must obey “all California driver requirements applicable to public roads.” The DMV is working on rules for the testing and deployment of driverless vehicles, but they will not be ready until late this year at the earliest.

In an effort to sidestep that set of limitations, Kentley-Klay got in touch with Randy Iwasaki of GoMentum Station, an old naval weapons base near San Francisco in March 2016. Iwasaki now calls the base the largest secure autonomous vehicle test facility in the world. According to documents obtained via public records request, Kentley-Klay wrote to Iwasaki, saying: “We do indeed have a vehicle to test, so now would be a great time to connect.” Iwasaki later reported that Zoox was “looking for a high-speed testing facility for their robo-taxi.”

Unlike other firms developing automated vehicles, Zoox does not have a cash-rich parent company like Google, Uber, or a traditional car company backing it. But Zoox has a healthy pile of money. Investors poured in over $200 million in 2016 alone, bringing the company’s value to around $1.55 billion. Although one source says that Zoox is already planning for vehicle deployments and even a regional charging network, another says that the car is still in the prototyping phase. A third person who was given a ride in Zoox’s development vehicle last year agrees with that second assessment: “The vehicle was a fairly early prototype. It didn’t even have a shell and was open to the air. They are probably the best at fundraising but technology-wise it was not that impressive.”

Last October, Zoox applied for a new trademark: “Expand Possible.” It sounds rather cryptic, but nothing could better describe the vision of this ambitious, secretive start-up.

uber robocar flips on side

Uber Suspends Robocar Testing After Crash

Editor’s note: On Monday morning Uber announced that it had resumed testing of its self-driving cars in San Francisco and that it would resume testing in Pittsburgh and Tempe, Ariz., later in the day, according to Bloomberg.

Uber has temporarily suspended the testing of its self-driving cars following the crash of one of them in Tempe, Ariz. The ride-hailing company had also been conducting experiments in Pittsburgh and San Francisco.

"There was a person behind the wheel [of the Uber car]," Tempe police information officer Josie Montenegro told Bloomberg. "It is uncertain at this time if they were controlling the vehicle at the time of the collision." She added that the crash had apparently been caused by another vehicle’s failure to yield.

The crash was no mere fender-bender: The Uber car—a specially equipped Volvo—was left lying on its side. Contrast the evident violence of that accident with the record racked up by Waymo, the spinoff of the Google car project, which has had only a few minor scrapes during its years-long testing.

A month ago Waymo accused Uber of purloining aspects of its robocar technology, in part by hiring Waymo engineers, notably Anthony Levandowski. Waymo claims that Levandowski brought along gigabytes-worth of Waymo’s plans for lidar sensing systems.

NuTonomy, which is also investigating robocars for a ride-hailing service, yesterday told the Boston Herald that it would continue its testing in South Boston. NuTonomy has another such a program in Singapore.

A woman behind the wheel of a car looking at a smartphone

Eyesight Technologies Will Watch You Drive, and That's a Good Thing

An alarming spike in traffic deaths on U.S. roads has been blamed on app-addled drivers, and computer vision firms say they have a remedy: They’ll save us from ourselves by assessing our alertness, mostly by figuring out which way we’re looking.

To stand out from the herd, Eyesight Technologies, of Herzilya, Israel, is touting its product as doing that and more besides.

“We don’t just look at the driver’s gaze, but also at his gestures,” says Iain Levy, head of the decade-old embedded vision company’s new automotive division. 

“Voices may differ between users and accents,” he adds, “but gesture is very natural. And as we move to higher levels of vehicle autonomy, gestures become even more interesting‚ as you have time to do more things than just drive the car.” Even so, the company is collaborating with a voice-recognition firm in order to allow oral communication as well.

The only hardware requirements are a camera and an infrared lamp. That way, no matter how bright it may be outside, the system can still track the driver’s eyelids, his iris, and the tilting of his head.

Here’s how the various functions might work together:

Read More
baidu's selfdriving car

Hackers Try to Steal Self-Driving Tech From China's Baidu

Robocar technology has received perhaps the most honest compliment mankind can bestow: Hackers tried to steal it. Even more extraordinary was the willingness of the targeted company, China’s Baidu, to acknowledge the attempted heist.

"It's very difficult to know who employs them to do that, but we know someone tried to hire someone in the underground market to steal from us," said Ma Jie, the cybersecurity chief of the Beijing-based company, in an interview with Bloomberg.

He wouldn’t or couldn’t give further details. But there are plenty of suspects with the money needed for black-hat operations, including world-bestriding companies flush with cash. Not one has been found guilty in a court of law, but accusations have begun flying: Last month, Waymo, the spinoff of Google’s robocar project, accused a former employee of stealing megabytes’ worth of its data and for a rival operation at Uber. Uber denies any wrongdoing.

Baidu followed in the footsteps of Google by turning a specialty in search into a subspecialty in autonomous cars. Like most robocar outfits, it is testing the technology in California and forming partnerships with auto makers and tech firms. Among Baidu’s recent deals are its investment seven months ago in Velodyne, the lidar company; its collaboration with BMW; and its partnership with Nvidia, developer of a system-on-a-chip for robocars.

Baidu has demonstrated partially robotic cars, and it’s talking up a ride-hailing robotaxi service as early as next year and a rollout of truly self-driving cars in 2021. In this, too, it is very much in line with the industry, which is offering an ambitious, although increasingly divergent, timeline for the new technology. 

Earlier this month, at a conference in Germany, Volvo, Audi, Ford and BMW estimated they’d have cars that could drive themselves with human supervision by 2021. Nvidia said that it would be able to do that this year, and be able to field a totally self-driving car by 2025. Bosch was more pessimistic, saying that full autonomy might take several years longer to achieve. 

With all these companies competing for skills that take years to develop, the price of engineering talent has soared. And it’s no wonder that the temptation to steal those engineers’ work has also increased.

intel buys mobileye

Intel Buys Mobileye for $15 billion

Intel is buying Mobileye, the Israeli robocar firm, for $15.3 billion. It’s one of the largest robocar acquisitions in a two-year buying frenzy that has swept both the auto industry and the tech companies that want to eat its lunch.

Mobileye made its name selling machine vision systems for driver-assistance features, such as lane keeping and emergency stopping. Unlike many companies, notably Waymo, it has so far eschewed expensive lidar, choosing instead to depend on a single (“mono”) camera.  Mobileye has done work for most of the major car makers in the world; the most prominent—but by no means the largest—such relationship was with Tesla Motors, which ended with some acrimony last year.

Intel has thus bought itself not only a full suite of robocar technology but also wide-ranging contacts in the auto industry. Its newly established self-driving unit also incorporates a 15 percent stake, which Intel acquired last month, in Here, a mapping company that BMW, Daimler and Volkswagen bought from Nokia in 2015 for $2.6 billion.

Intel’s self-driving unit will be run by Mobileye management, in Jerusalem.

The deal validates Mobileye’s strategy of resisting the found-and-flip routine that most Israeli tech startups have followed (though that trend may now be changing). It held out for top dollar and it got it: Not only is $15 billion the third-largest market valuation of any publicly traded Israeli company, it is high for an auto-parts supplier and not unrespectable for an OEM.

Look at Mazda’s market capitalization, which today stands at $8.7 billion. Or at the $8 billion that Samsung paid in November for Harmon, the car-audio tech company. The only deal that comes to mind that was comparable in size was NXP’s $12 billion purchase of Freescale, an auto chip maker, back in 2015.

By comparison, Uber spent just $680 million last August to acquire Otto, a self-driving truck startup. At the time, Uber said that it valued the expertise of Otto’s staff, above all the veteran robocar expert Anthony Levandowski, a key founder of what is now Waymo.

Uber thus had to pay just 4.4 percent as much as Intel just did for what might be called a roughly comparable suite of robocar knowledge. Uber got quite a bargain—you might even say it was a steal.

Deep learning from the ground up helps Drive's cars handle the challenges of autonomous driving

How Is Mastering Autonomous Driving With Deep Learning

Among all of the self-driving startups working toward Level 4 autonomy (a self-driving system that doesn’t require human intervention in most scenarios), Mountain View, Calif.-based’s scalable deep-learning approach and aggressive pace make it unique. Drive sees deep learning as the only viable way to make a truly useful autonomous car in the near term, says Sameep Tandon, cofounder and CEO. “If you look at the long-term possibilities of these algorithms and how people are going to build [self-driving cars] in the future, having a learning system just makes the most sense. There’s so much complication in driving, there are so many things that are nuanced and hard, that if you have to do this in ways that aren’t learned, then you’re never going to get these cars out there.”

It’s only been about a year since Drive went public, but already, the company has a fleet of four vehicles navigating (mostly) autonomously around the San Francisco Bay Area—even in situations (such as darkness, rain, or hail) that are notoriously difficult for self-driving cars. Last month, we went out to California to take a ride in one of Drive’s cars, and to find out how it’s using deep learning to master autonomous driving.

Read More
Waymo is one of the companies that California rules would allow to operate and sell autonomous vehicles by the end of the year

California Gives the Green Light to Self-Driving Cars

In a bold attempt to regain its place as a self-driving pioneer, California today proposed regulations that would make it one of the first places in the world where autonomous vehicles could carry paying passengers without a licensed driver on board.

The new regulations, which could come into force as soon as November, depart from the cautious approach California has taken in the past. They would allow manufacturers to certify their own fully driverless vehicles as safe and effective without a detailed inspection, or even having to test in the state beforehand.

“You can apply for a permit to deploy when…you as a manufacturer believe the vehicles are ready to go,” says Bernard Soriano, Deputy Director of the California Department of Motor Vehicles (DMV). “The traditional system of not wanting to be sued crazy controls when manufacturers will do this.”

The latest regulations for the deployment of autonomous vehicles are noticeably looser than draft versions circulated over the past 15 months. Initially, the DMV suggested a certification process that required a demonstration of the technology to a third-party testing organization, a licensed driver in the vehicle at all times, and a time-limited permit granted on the condition that the manufacturer file monthly progress reports. The California rules also demanded that manufacturers maintain ownership of the vehicles.

At one point, the DMV even considered requiring companies to seek permission from police departments and local governments in every town or city their cars might drive themselves through. “But if you have a vehicle designed to go through three different jurisdictions, that would be a tough task,” admits Brian Soublet, Chief Counsel at the DMV. “Instead, we’re putting the onus on the manufacturer.”

The new regulations sweep away many of the original requirements. Manufacturers do not need to have previously held a testing permit, nor will they have to make their technology available for independent testing. Once they have secured an operational permit, they will be free to sell their vehicles to whomever they please, and will not need to file any reports of operational problems with the DMV.

Testing has been simplified, too. Companies that want to test highly automated Level 4 and Level 5 vehicles, like Uber’s self-driving taxis, will now be allowed to pick up passengers, although not in exchange for payment. Crucially, they will not have to ask permission from every city in which they want to operate driverless cars. Instead, they only have to notify local authorities that they are doing so.

But the new regime is not a complete free for all. Manufacturers will have to certify that their vehicles comply with existing federal motor vehicle safety standards, or at least have an exemption from them. This could be a time-consuming process for cutting-edge driverless cars that lack traditional steering wheels, accelerator and brake pedals, such as those being developed by Google spin-out Waymo and start-up Zoox.

“The DMV’s rules are also going to shift a big part of the conversation to the federal level,” says Bryant Walker-Smith, a law professor at the University of South Carolina. “Some federal safety standards, such as one addressing brake pedals, arguably conflict with a truly driverless car. The National Highway Traffic Safety Administration (NHTSA) could change these standards, but that might take too long. NHTSA also has authority to grant exemptions from the federal safety standards, but this authority is limited.”

Level 4 and 5 vehicles will also need a law enforcement interaction plan that instructs police officers, fire fighters and paramedics how to deal with the vehicle in the event of a break-down or accident. Importantly, the DMV also requires all such autonomous vehicles to have a dedicated communications link with a remote operator, who must be able to provide information on the vehicle’s location and status, and have two-way communication with any passengers. The operator could even teleoperate the car in an emergency.

The new rules also empower the DMV to crack down on companies whose advertising implies that vehicles are autonomous when they are not—a clear signal to companies such as Tesla and Mercedes-Benz, whose marketing has caused confusion in past.

California is not the first jurisdiction to pass rules governing the deployment of fully automated vehicles. Michigan has a law contemplating driverless fleets, and Florida has a law that its drafter says covers this, too. “But this would make California the most consciously permissive jurisdiction in the world,” says Ryan Calo, a professor at the University of Washington who teaches a course on robot law. “I question the wisdom of self-certification, especially with players that are not as sophisticated. I think it would be wiser to have third parties audit the technology.”

John Simpson of nonprofit technology advocacy group Consumer Watchdog has other concerns. “One of the things that made sense before was that if you were going to deploy a Level 4 or 5 vehicle, you had to do a year of testing and provide a disengagement report,” he says. “This is a terrible change and a substantial weakening of the regulation.”

The regulations could still evolve further. A 45-day public comment period starts today, followed by a public hearing in late April. “There’s always some room to wiggle when there’s a comment period,” admits Soriano. If everything goes smoothly, the rules could go into effect as soon as November.

The original 1982 ItalDesign Capsula concept car, a strange dark grey car

What's Edgier Than a Flying Robocar?

The only thing edgier than a flying robocar is half a flying robocar. That’s the concept that Italdesign and Airbus are to unveil this week at the Geneva Auto Show.

Yes, yes, you’re thinking that now you’ve seen everything. Except that you haven’t seen it yet. As for us here at IEEE Spectrum, we’ll believe it when we see it. 

And, no, this flying car has nothing to do with that other Airbus flying-vehicle project, the Vahana autonomous air taxi. That’s the one being developed by A3, the airplane manufacturer’s Silicon Valley subsidiary. As our recent posts attest, we aren’t too enthralled with Vahana, nor with the Ehang air taxi that’s being readied for use in Dubai.

The Italdesign-Airbus concept would assign the flying job not to the vehicle itself but to a large drone—one measuring some 5 meters by 5 meters, according to Automotive News. When you’re mired in traffic, you’d just call on that pterodactyl, which would come and swoop down, grab the upper half of the robocar and waft it and you away. Only the upper half—the passenger cab—gets lifted: the bottom half—the drivetrain and wheels—stays put, inching its way through traffic, and eventually finding its way home.

This concept is just another application of the layer-cake design of the Capsula, a 1982 thought experiment by Italdesign. The bottom layer carries the automotive guts—chassis, engine, brakes, wheels, even the spare wheel—leaving space on top for different “capsules”. Depending on the capsule you choose, you could get a sedan, light truck, police car, ambulance, or any other vehicle.

And to what destination, pray tell, would the big drone carry you? Maybe you’d choose from among many spare bottom halves predeployed all around your city. Then the drone drops you and the upper layer onto one. Presto! You’re a car again.

And if your newly assembled layer cake of a car gets stuck in traffic, you could call on the drone to rescue you yet again. So could all the other people, until at last the road network got so chockablock with aimlessly moving bottom halves that there’d be no choice but to get airborne and stay that way.

The rear end of a white car with a black knob on top and sign that says

Waymo's Fight With Uber Might Be the First Shot in a Self-Driving Car IP War

Waymo filed a lawsuit yesterday accusing Uber of stealing the secret designs of circuit boards and laser ranging lidars used in its self-driving cars. Like all legal complaints, it ends with a long wish list of “reliefs” it wants the federal court in California to deliver, but the ramifications of this lawsuit could stretch far beyond financial damages or legal costs.

“For years I’ve warned about a potential automated driving patent war that could rival the notorious smartphone patent war,” says Bryant Walker-Smith, a law professor at the University of South Carolina and an expert in self-driving regulations. As autonomous vehicles transition from amusing gimmicks to money-making products, who controls the key intellectual property could determine which companies thrive and which fall by the wayside.

Read More
A man, Anthony Levandowski, in a gray shirt holding a microphone speaking in front of a silver sedan.

"Stop, Thief!" Waymo Screams at Uber

Google’s self-driving car spinoff, Waymo, is charging a former employee, now at Uber, with stealing trade secrets by the gigabytes—claiming damages of at least US $500 million. What’s more, Waymo reconstructs the alleged crime in amazing detail, including an attempt by the main alleged perpetrator to cover his tracks.

Waymo says in a blog post that the secrets involve its lidar sensing system, the crown jewel in its self-driving design. Waymo recently announced home-designed lidar sets with overlapping ranges and functions, the first such system in the self-driving business.

“Recently, we received an unexpected email,” Waymo says. “One of our suppliers specializing in lidar components sent us an attachment (apparently inadvertently) of machine drawings of what was purported to be Uber’s lidar circuit board — except its design bore a striking resemblance to Waymo’s unique lidar design.”

And the chief thief, Waymo says, is Anthony Levandowski. Levandowski is listed as an inventor on many Google lidar patents, and Google’s acquisition of his startup, 510 Systems, was foundational to the self-driving car effort that became Waymo. 

Read More

Cars That Think

IEEE Spectrum’s blog about the sensors, software, and systems that are making cars smarter, more entertaining, and ultimately, autonomous.
Contact us:

Philip E. Ross
New York City
Willie D. Jones
New York City
Senior Writer
Evan Ackerman
Washington, D.C.
Mark Harris

Newsletter Sign Up

Sign up for the Cars That Think newsletter and get biweekly updates, all delivered directly to your inbox.

Load More