Carnegie Mellon's Red Team went into the 2005 DARPA Grand Challenge as the favorite to win. They'd led the pack in the 2004 event, and had been successfully running their two heavily modified autonomous Humvees, H1ghlander and Sandstorm, on mock races across the desert for weeks without any problems. When H1ghlander set out on the 212 km (132 mi) off-road course at dawn on 8 October 2005, it led the pack, and gradually pulled away from Stanford's robot, Stanley.
About two hours into the race, however, H1ghlander's engine began to falter, causing it to struggle in climbs and never reach its top speed. Nobody could tell what the issue was, but it slowed the vehicle down enough to cost it more than 40 minutes of race time. Stanley passed H1ghlander and went on to win the race by just 11 minutes. Even after the event, CMU wasn't able to figure out exactly what happened. But last weekend, at an event celebrating the 10th anniversary of the DARPA Urban Challenge (which CMU won handily with their autonomous Chevy Tahoe BOSS), they accidentally stumbled onto what went wrong.
Here’s the point in the race where H1ghlander started to falter; it’s part of a fantastic NOVA documentary on the DARPA Grand Challenge which you should watch in its entirety if you have time:
Even the DARPA Grand Challenge winner, Stanford University, seemed a bit surprised by how things turned out. "It was a complete act of randomness that Stanley actually won," Stanford team lead Sebastian Thrun later said. "It was really a failure of Carnegie Mellon's engine that made us win, no more and no less than that."
Here are some excerpts from Red Team's race logs recorded immediately following the Grand Challenge:
October 11: The root cause that capped H1ghlander's speed and crippled its climbs is not yet known. Requested speeds above 20 mph were under-achieved, even on the long, straight, level roads. H1ghlander didn't even reach intended speeds going downhill. H1ghlander apparently stopped, rolled backwards, then re-climbed a few times. Weak climbing and stopping are not great practices for winning races. The capped speeds and weak climbs cost H1ghlander over 40 minutes of schedule time. The root cause is still a mystery.
October 12: H1ghlander’s engine was observed to be shaky immediately following the race. The first indication of possible engine trouble was observed when driving H1ghlander from the finish line to the inspection area with a human at the wheel. The engine was running very rough and almost died repeatedly in just that 50 yards of driving with a human foot on the accelerator pedal.
This engine problem is unlike any one that we have seen in the past, as engine performance is severely degraded at and anywhere near idle. Data indicated no limp home mode, no safety mode, and no low-torque mode. Detailed fuel, oil and transmission samples will be analyzed. We do not yet know the root cause that slowed H1ghlander’s driving on race day.
It turned out that the fuel was okay. The oil and transmission fluid were also okay. The electrical system was fine too. With the DARPA Urban Challenge up next and a completely new vehicle under development for that, the CMU team moved on.
BOSS on display at CMU last week, with H1ghlander in the backgroundPhoto: Evan Ackerman/IEEE Spectrum
Last week, CMU celebrated the 10th anniversary of BOSS' DARPA Urban Challenge win in 2007. BOSS, Sandstorm, and H1ghlander were all pulled out of storage at CMU and tidied up a bit to be put on display. As H1ghlander's engine compartment was being cleaned with the engine running, Spencer Spiker (CMU's operations team leader for the DARPA challenges) leaned against the engine with his knee, and it started to die. This little box is what he was leaning against, as shown to Clint Kelly (who directed DARPA's research programs in robotics and autonomous systems in the 1980s) by CMU Red Team leader William "Red" Whittaker:
Photo: Evan Ackerman/IEEE Spectrum
The box is a filter that goes in between the engine control module and the fuel injectors, one of only two electronic parts in the engine on a 1986 Hummer. Spencer discovered that just touching the filter would cause the engine to lose power, and if you actually pushed on it, the engine died completely. But, from a cold start, if the filter wasn't being touched, the engine would run fine. There was nothing wrong with H1ghlander's sensors, or software: this filter cost H1ghlander 40 minutes of race time, and the win. "How about that, buddy!" Red said to Chris Urmson (who was working on perception for Red Team during the DARPA challenge, and ran Google's self driving car program for seven years before starting his own autonomous vehicle company) at the CMU event, showing him the filter. "You're off the hook!"
As to what may have caused this hardware failure in the first place, many team members at the CMU event suggested that it may have happened just a few weeks before the Grand Challenge, on September 19, when H1ghlander got into a bit of an accident after a 140-mile autonomous test:
Here's an excerpt from a blog post by Vanessa Hodge, who worked on vehicle navigation and was following H1ghlander in a chase car that night:
H1ghlander was driving autonomously back to the entrance road so we could drive it back to the shop to pamper it before the race. We came to a part of the trail where there was a swamp on the left and a boulder-ridden mountain side on the right, with a road width a little bit larger than the vehicle. H1ghlander kicked up some thick dust and I slowed down to a stop to let the dust settle before catching up. The team in the chase car watched our vehicle display which monitors its actions while the dust settled. Problems appeared in the display, and one team member immediately hit the emergency pause button, but it was too late. In the second we lost visual, H1ghlander tracked off to the right of the path up the slope, slid on its side and flipped entirely to the other side.
By September 22, H1ghlander was back up and running, "hot and strong." But, that filter may have taken some damage that was difficult or impossible to diagnose, and it ended up failing at the worst possible time.
While it's impossible to know how a DARPA Grand Challenge win by H1ghlander might have changed autonomous car history, the people I spoke with at CMU generally seemed to feel like everything worked out for the best. BOSS had a convincing win at the DARPA Urban Challenge in 2007, and Stanley's performance at the Grand Challenge helped to solidify Stanford's place in the field. Roboticists from both CMU and Stanford helped to form the core of Google's self-driving car program in California, and today, Pittsburgh is one of the places where both established companies and startups come to do self-driving research, development, and testing. There are very few lingering feelings about what happened, and everyone involved has long since moved on to bigger and better things. But all the same, it's nice that at last, this final mystery has been solved.