Everybody likes watching robots fall over. We get it, it’s funny. And we here at IEEE Spectrum are as guilty as anyone of making it a thing: Our compilation of robots falling down at the DARPA Robotics Challenge eight years ago has several million views on YouTube. But a couple of months ago, Agility Robotics shared a video of one of its Digit robots collapsing while stacking boxes during the ProMat trade show, which went nuts across Twitter, TikTok, and Instagram. Agility eventually issued a statement to the Associated Press clarifying that Digit didn’t deactivate itself due to the nature of the work, which is how some viewers reacted to the viral clip.
Agility isn’t the only robotics company to share its failures with an online audience. Boston Dynamics, developer of the Spot and Atlas robots, may have been the first company to be accused of “robot abuse” because of its videos, and the company frequently includes footage of its research robots being unsuccessful as well as successful on YouTube. And now with 1,100 Spots out in the world being useful, falls happen both more frequently, and more visibly.
Even though falling robots aren’t a new thing, what may be a new(ish) thing are some technological advances that have changed the nature of falling. First, both Boston Dynamics and Agility Robotics have human-scale bipedal robots for which not falling seems pretty normal. This is a relatively recent development. Although a number of companies are working on humanoids, the Agility and Boston Dynamics humanoids are (as far as we are aware) the only ones that can routinely handle untethered dynamic walking.
“Sometimes the robot is going to break something when it falls. But it’s learning, and eventually I think these robots will fall even less often than people do.” —Jonathan Hurst, Agility Robotics
The other important advance is that these humanoid robots are usually able to fall without destroying themselves. During the DARPA Robotics Challenge in 2015, falling generally meant doom for the competitors, with one exception: Carnegie Mellon University’s CHIMP, which was built like a literal tank. Since then, roboticists have tried adding things like armor and airbags to keep a falling robot in one piece. But now, these robots can fall with minimal drama and get back up again. If they do suffer damage, they can be easily fixed.
And yet, even though falling has become much less of a big deal for the roboticists, it’s still a big deal for the general public, as these viral videos of robots falling down prove. We recently spoke with Agility Robotics’ chief robot officer, Jonathan Hurst, and head of customer experience, Bambi Brewer, as well as Boston Dynamics CTO Aaron Saunders to understand why that is, and whether they think things are likely to change anytime soon.
Boston Dynamics’s Aaron Saunders, and Agility Robotics’ Jonathan Hurst and Bambi Brewer on...
- Why people react so strongly to robots falling
- How bad is falling for robots anyway?
- How can robots learn to fall better?
- How can robots learn to fall safely?
- What should people know about falling robots?
Why do you think people react so strongly to seeing robots fall over, especially bipedal robots?
Jonathan Hurst: People post funny videos of pets or kids making some expression or having a reaction that you can identify with. It’s even funnier when it’s a robot that wouldn’t typically do that. And so when Digit [at ProMat] seems to be just like, “I’m so tired of doing this work” and falls down, people are like, “I understand you, robot!” But [seeing robots behave that way] is going to become more common, and when people see this and it becomes just a regular part of their experience, the novelty will wear off.
Bambi Brewer: People who make robots spend a lot of time trying to present them at their best. The way robots move does seem very repetitive, very scripted. I can see why it’s very interesting when something goes wrong, because the public usually doesn’t see what that looks like, and they’re not used to those moments yet.
“People perceive machines based on how they perceive themselves. Falling on its face is a good example of something that looks bad for a robot but might not actually be bad.” —Aaron Saunders, Boston Dynamics
How different is falling for robots than for humans?
Hurst: The way I think about the robot right now is like a two-and-a-half-year-old child. They fall more often than adults do, and it’s not terribly concerning. Sometimes they skin their knee. And sometimes the robot is going to break something when it falls. But it’s learning, and eventually I think these robots will fall even less often than people do. Physics is still true, though, and so it’s probably going to be on the same order of magnitude as how often people fall. It won’t be rare.
When you think about this ‘physics is true’ thing—that’s actually where robots will be able to have superhuman capabilities. A robot is going to be close to human strength and close to human speed, but you can take much bigger risks with a robot because you don’t really care that much if you break something.
Fundamentally, I don’t care if the robot breaks. I mean, I care a little bit, but I care a lot if any of our employees were to fall.
Do you think that humanoid robots falling in nonhuman ways might be part of why people react so strongly to these videos?
Aaron Saunders: We have a massive metal frame around the front of Atlas. It’s okay if it face-plants. It tucks its limbs in to protect them and other parts of the robot. A human would do the opposite—we put our limbs out and try to protect our heads. Robots can handle certain types of impacts and forces better than humans can. We have a lot of conversations around how people perceive machines based on how they perceive themselves. Falling on its face is a good example of something that looks bad for a robot but might not actually be bad.
“I can see why it’s very interesting when something goes wrong, because the public usually doesn’t see what that looks like, and they’re not used to those moments yet.” —Bambi Brewer, Agility Robotics
How normal is it for your robot to fall?
Saunders: Almost everything we do on Atlas is about pushing some limit. We don’t shy away from falling, because staying in a safe place means leaving a lot on the table in terms of understanding the performance of the machine and how to solve problems. In our development work, it falls all the time, both because we’re pushing it and because there’s very little risk or hazard—we’re not delivering Atlas out into the world.
On a long flat sidewalk, I don’t think Atlas would fall in a statistically relevant way. People think back to the video of robots falling all over the place at the DARPA Robotics Challenge, and that’s not the type of falling we worry about now.
For Spot, falling can be more of a risk, because it is out in the world. On a weekly basis, our internal fleet of Spots are walking about 2,000 kilometers, and we also have them in these test cells where they’re walking on rocks, on grates, over obstacles, and on slippery floors. We want to robustly test all of this stuff and try to drive those cases of falling down to their minimums.
“If a person is carrying a baby and falls down some stairs, they have this intuition and natural ability to save the baby, even if it means injuring themselves. We can design our robots to do the same kind of thing to protect the people around it when it falls.” —Jonathan Hurst, Agility Robotics
How big of a deal is it for your robot to fall?
Hurst: Digit was designed to fall. That’s one of the reasons that it has arms—to be able to survive a fall. When we were first designing the robot, we said, okay, at some point the robot’s going to fall, how can we protect it? We calculated how much padding we would need to minimize the acceleration on the electronic components. It turned out that we would have needed several inches of padding, and Digit would have ended up looking like the Michelin Man.
The only realistic way to have Digit safely decelerate was to have an appendage that’s going to stick out and absorb that fall. And where is the best place to locate that appendage? You get the same answer as you do when you think about inertial actuation and bimanual manipulation. Digit’s arms are where they are not because we’re trying to build a humanoid, but because we’re trying to solve locomotion challenges, manipulation challenges, and making sure that we can catch the robot when it falls.
Was there a point during the development of your robot where falling went from normal to unusual?
Saunders: The thing that really took us from worrying about normal walking to feeling pretty good about normal walking is when we pushed aggressively into things that went way beyond walking.
To jump and land successfully, we needed to develop control algorithms that could accommodate all of the mass and the dynamics of the robot. It was no longer about carefully picking where you put your foot for each step, it was about coordinating all of that moving mass in a really robust way. So when Atlas started jumping and doing parkour, it made walking easier too. A few weeks ago, we had a new team member go back and apply some of the latest control algorithms that we’re using for parkour to our standing algorithm. With those new algorithms we saw big improvements in the robot’s ability to handle disturbances from a stand—if somebody were to shove the robot, this new controller is able to think and reason about all of its dynamics, resulting in massive gains in how Atlas reacts.
“We need to give a very clear signal to people to tell them not to try and help—just step back and let the robot fall. It’ll be fine.” —Bambi Brewer, Agility Robotics
At this point, how much is falling just an “oops,” and how much is it a learning opportunity?
Hurst: We’re always looking for bugs that you can iron out. Digit’s collapse at ProMat was one. In this scenario, there really should not have been an emergency stop.
Brewer: Falls are points at which somebody is filing a bug card, or looking through the logs. They’re trying to figure out what happened, and how to make sure it doesn’t happen again. At ProMat, there was something wrong with an encoder in the arm. It’s been updated now. It was a bug that hadn’t occurred before. Now if that happens, the robot’s arm will freeze, but the robot will remain upright.
Saunders: On Spot, I think there are relatively few learning opportunities these days. We know pretty well what Spot’s capable of, in what situations a fall might occur, what the robot is likely to do in those situations, and how it’s going to recover. We designed Spot to be able to fall robustly and not break, and to get up from falls. Obviously, there are some extreme cases—one of our industrial customers had a need for Spot to cross a soapy floor, which is about as close as you can get to walking on ice, a challenge for anything with legs. So our control team set up a slippery environment in our lab, using cooking oil on plastic, and then just started “robustifying.” They figured out how to detect slips and adapt the gait of the robot, and went from a situation where falling was regular to one where falling was infrequent.
For Atlas, generally the falling state happens after the part that we care about. What we’re learning there is what went wrong right before the fall. If we’re working on one of Atlas’s aerial tricks—say, something that we’ve never landed before—then of course we’re doing a ton of work to figure out why falls happen. But if we’re just walking around the lab, and there was some misstep, I don’t think people stress out too much, and we just stand it back up and reset it and go again.
“Robots should be able to fall. We should give them a break when they do.” —Aaron Saunders, Boston Dynamics
We’re not afraid of a fall—we’re not treating the robots like they’re going to break all the time. Our robot falls a lot, and one of the things we decided a long time ago that we needed to build robots that can fall without breaking. If you can go through that cycle of pushing your robot to failure, studying the failure, and fixing it, you can make progress to where it’s not falling. But if you build a machine or a control system or a culture around never falling, then you’ll never learn what you need to learn to make your robot not fall. We celebrate falls, even the falls that break the robot.
If a robot knows that it’s about to fall, what can it do to protect itself, and protect people around it?
Hurst: There are strategies when you know you’re about to fall. If a person is carrying a baby and falls down some stairs, they have this intuition and natural ability to save the baby, even if it means injuring themselves. We can design our robots to do the same kind of thing to protect the people around it when it falls.
Brewer: In addition to the robot falling safely, we need to give a very clear signal to people to tell them not to try and help—just step back and let the robot fall. It’ll be fine.
Hurst: The other thing is to try to fall sooner rather than later. If you’re not sure whether you can stay balanced, you might end up taking a step to try to correct, and then another step, and then maybe you’re moving in a direction that’s not all that controlled. So when it starts to lose its balance, we can tell the robot, “Just fall. You’ll get back up.”
Saunders: We have these detections inside of our control system that trigger when the robot starts doing something that the controller didn’t ask it to do. Maybe the velocity is starting to do something, or the robot is at some angle that it isn’t supposed to be. If that makes us think that a fall might be happening, we’ll run a different controller to try to stop it from falling—Atlas might decide to swing its arms, or move its upper body, or throw its leg out. And if that fails, there’s another control layer for when the robot is really falling. That last layer is about putting the robot in a state that sets its pose and joint stiffnesses to basically ensure that it will do minimal damage to itself and the world. How exactly we do this is different for each robot and for each type of fall. If you comb through videos of Atlas, you might see the robot tucking itself up into a little bit of a ball—that’s a shape and a set of joint stiffnesses that help it mitigate impacts, and also help protect things around it.
Sometimes, though, these falls happen because the robot catastrophically breaks. With Atlas, we definitely have instances where we break the foot off. And at that point, I don’t have good answers.
The next time a video of a humanoid robot falling over goes viral, whether it’s your robot or someone else’s, what is one thing you’d like people watching that video to know?
Hurst: If Digit falls, I think it’d be great for people to know that the reaction from the engineers who built that robot would not be, “Our robot fell over and we didn’t expect that!” It would just be a shrug.
Brewer: I’d like people to know that when a robot is actually out in the world doing real things, unexpected things are going to happen. You’re going to see some falls, but that’s part of learning to run a really long time in real-world environments. It’s expected, and it’s a sign that you’re not staging things.
Saunders: I think people should recognize that it’s normal for equipment to sometimes fail. Equipment can be fixed, equipment can be improved, and over time, equipment gets more and more reliable. And so, when people see these failures, it may be a situation that the robot has never experienced. They should know that we are gathering all that information and that we’re continuously improving and iterating, and that what they’re seeing now doesn’t represent the end state. It just represents where the technology is today.
I also think that there has to be some balance between our expectations for what robots can do, and the process for getting them to do it. People will come to me and they’ll want a robot that can do amazing things that robots don’t do yet, but they’re very nervous if a robot fails. If we want our robots to do amazing things and enrich our lives and be our tools in the workforce, we’re going to need to build those capabilities over time, because this is emerging technology, not established technology.
Robots should be able to fall. We should give them a break when they do. It’s okay if we laugh at them. But we should also work hard to make our products safe and reliable and things that we can trust, because if we don’t trust our robots, we won’t use them.
- Life-Size Humanoid Robot Is Designed to Fall Over (and Over and Over) ›
- Robots Learning Judo Techniques to Fall Down Without Exploding ›
- Robot Learns Human Trick for Not Falling Over ›
- Video Friday: Robot vs. Door ›