DARPA Robotics Challenge Trials: Day 2 and The Future

The DARPA Robotics Challenge Trials ended on Saturday night, and we're heading home from Florida today. If you haven't been following the event, you can watch recordings of most challenges here, and below is DARPA's Day 2 wrap video: 

We're going to have some detailed follow-ups on some DRC specifics over the next few weeks, but for now, we'll leave you with our thoughts from Day 2 of the competition, as well as some comments from DARPA about what we have to look forward to at the DRC Finals next year.

SCHAFT was definitely the team to beat

SCHAFT totally rocked the DRC Trials. Enough said. We'll say more, of course, because that's what we do, but look for it in a dedicated article later in the week.

 

SCHAFT may have won, but lots of teams did very, very well

At the initial DRC press conference on Friday morning, Gill Pratt told us how impressed he would be if any team managed to score half of the points. Four teams cracked the halfway mark, with three more in double digits. We initially made some fairly pessimistic projections as well, and we certainly came away impressed.

 

At this point, it's more about software than hardware

All of the robots in the DRC were presumably capable, from a hardware perspective, of completing the entirety of all of the tasks. The reason that they couldn't, however, came down to software and control, or in other words, making the robot do exactly what its operators wanted it to do. We're expecting a huge improvement in software over the next year, which will make these robots much more capable.

 

It's also more about experience than anything else

Practice makes perfect for robots too. Part of the reason that SCHAFT and some of the other teams did surprisingly well at many tasks was that they spent lots of time practicing them over and over again. With a year before the DRC Finals, we expect many teams to be significantly more competitive.

 

More sensors aren't necessarily more effective, at least until partial autonomy becomes necessary in 2014

CHIMP (and each ATLAS robot), for example, has a ton of sensors on its head, but the limiting factor for teams was the amount of bandwidth that they had to work with. With onboard decision-making, you can take advantage of more data, but otherwise, you can get away with just a few simple sensors. That said, DARPA may make autonomy a more important factor in 2014, and then having additional sensors will make a difference. 

 

Spectators seemed to understand and appreciate the pace of the DRC

We were a little bit worried that normal humans (i.e. non-roboticists) wouldn't really understand why the DRC robots kind of look like they're terrible most of the time. But, it was heartwarming to hear the spectators cheer enthusiastically when, after five or 10 minutes of pondering, a robot like THOR would successfully take a step.

 

Robots are more fun when they have personality

This is an entirely subjective thing and might have nothing to do with the DRC itself, but I loved the fact that some of the DRC robots were entertainingly active on Twitter all weekend. WPI's Warner might be our favorite example, as it apparently developed a little bit of a crush on JSC's Valkyrie (totally understandable). We here at IEEE Spectrum cannot confirm or deny whether anything actually, you know, happened.

 

NASA JSC's Valkyrie is in this for the endgame

It's easy to look at the DRC Trials final scores and come to the conclusion that SCHAFT was the absolute winner (scoring 27 out of 32 possible points) and JSC's Valkyrie (which scored 0 points!) was the biggest disappointment. And although I won't mention any names (ahem), lots of people are already portraying the results that way. We'll have a much more in-depth article on this in the future, but what you have to consider is that SCHAFT built their robot on an existing, proven platform that they have a huge amount of experience with, while JSC built Valkyrie more or less from scratch, and therefore didn't have a lot of time to get their software finalized, much less practice the tasks. Again, we'll get into all of this in a dedicated article, but it would be hasty to dismiss Valkyrie based on the DRC Trials scores.

 

"Gaming" the tasks might be okay

In the debris clearing task, the intent was for the robots to remove individual pieces of wood, followed by a metal truss, from a hallway, and then pass through it. A few teams like RoboSimian (pictured above) figured out that it would be much easier to to just drag the metal truss out of the hallway completely, taking all of the wood with it, no grasping involved. This was not the intent of the task, but to extrapolate to a real disaster scenario, the intent is not at all important, while the results are, and if there's an easier way to do things, that's the way to do 'em. (RoboSimian did not score in this task because it ran out of time, but other teams that used this approach did receive points from DARPA.)

 

Even successful robots often barely succeeded

Many of the DRC robots were able to complete many of the tasks, but it's important to remember that these tasks are very far from the sort of random events that a robot would find in an actual disaster. Teams took advantage of the setup time that they were allotted, sometimes using tape measures to carefully position their robots in a specific starting location. Robots are generally terrible at randomness, and the DRC Trial tasks weren't random at all. Disasters themselves are (by definition) random, so we'll see how much of that gets added to the finals in 2014.

 

The DRC Finals are going to be incredible

Very incredible. That is all.

 


 

The Future

DARPA hasn't yet made any official statements about the DRC Finals, but during the press conference on Saturday night, we got a chance to ask Gill Pratt whether the excellent overall performance that the robots put on over the weekend might lead to a potentially more challenging DRC Final:

I think we can make the finals a little harder than I had thought. I had a dream for really, really hard finals that all of my staff talked me out of, so somewhere in the middle there is sort of where it will be. But, we have been in touch with the teams from the beginning. The teams that we fund, we got proposals from them, so we had our eyes on how well they were doing. Six months after they started work, we had critical design reviews: we went and visited each of the teams everywhere in the world that were being funded. And, we had the virtual robotics challenges, where we saw how good the software worked, from all of those teams plus many others. That really helped us to calibrate how difficult to make the challenge itself. We erred a little bit on the side of being conservative because we wanted the teams to do well, but we made sure there was enough of what we call "dynamic range" in the scoring so no team would get all of the points and we would get end up with on ties on the top. So I think you’re right, we can make the finals a little bit more difficult than we had thought. We’re not quite sure how much yet, but whatever we think now, we will be checking in with the teams several times within the coming year to see just how hard to make it.

Other things we learned at the Saturday night press conference, presented in some easy to digest InfoNuggets:

  • The eight winning teams will (pending contract stuff) probably get $1 million each in funding from DARPA.
  • The date and location for the finals has not yet been set.
  • Tasks for the finals will be modified and combined into "a more authentic mission for disaster response."
  • Teams will have options in the finals, like a variety of tools to choose from.
  • DARPA wants ATLAS to be tetherless for the finals.
  • There will be qualifiers for the finals, and non-funded teams that didn't participate in the trials are welcome to try out for the finals.
  • After 2014, there's a possibility that Japan will host further robotics disaster challenges.

[ DARPA Robotics Challenge ]

Advertisement

Automaton

IEEE Spectrum's award-winning robotics blog, featuring news, articles, and videos on robots, humanoids, automation, artificial intelligence, and more.
Contact us:  e.guizzo@ieee.org

Editor
Erico Guizzo
New York, N.Y.
Senior Writer
Evan Ackerman
Berkeley, Calif.
 
Contributor
Jason Falconer
Canada
Contributor
Angelica Lim
Tokyo, Japan
 

Newsletter Sign Up

Sign up for the Automaton newsletter and get biweekly updates about robotics, automation, and AI, all delivered directly to your inbox.

Advertisement