Every time we post about autonomous delivery drones, we have to point out that despite the promises implied byoverproduced and optimistic videos, the drones are simply not capable of autonomous navigation in complex environments. Same goes for thosecameradronesthat promise to follow you: the videos inevitably show them following skiers on wide open slopes, surfers on the wide open sea, or other people doing things very far away from inconvenient obstacles like trees.
So far, we’ve only seen a tiny handful of drones capable of dynamically detecting and avoiding obstacles at a useful speed. Qualcomm and UPenn have been working on some, and MIT has that speedy tree-avoiding fixed-wing drone.
A Silicon Valley company called Skydio, founded by a team of researchers from MIT and Google X’s Project Wing, have posted a video that shows a drone following people jogging and biking while autonomously avoiding tree trunks and branches. In other words, it’s what other drone companies have been promising to be able to do for years, except Skydio actually delivers—or at least it shows real progress (and footage) toward that goal.
This drone, with some kind of crazy camera hardware on it, is following that dude on the bike while dynamically avoiding obstacles that show up in front of it basically out of nowhere. It has very little time to react, and it also has to avoid stuff in such a way that it can keep going in the direction that it needs to go.[shortcode ieee-pullquote quote=""Ultimately, all the information a drone needs to be good at its job is in the images. The challenge is extracting it and using it. Full scene understanding, deeper context awareness, learning based on user data and feedback, will all unlock more powerful autonomy."" float="right" expand=1]
We’re usually very skeptical about drones doing amazing things in videos, expecting to see localization systems lurking in the background, or the obvious polish of careful editing and multiple cameras and takes. Skydio’s video looks (surprisingly) like they just took their drone out to the woods and recorded it doing its thing. Just to be sure, we asked Skydio’s Adam Bry to confirm for us that there are no shenanigans, and there aren’t. “All the footage is just taken from our normal testing,” Bry told us. “Nothing was staged and it’s all fully autonomous. All of the navigation is done entirely based on a multi-camera array with all computation done onboard on a state of the art mobile CPU.”
This is very cool stuff, but it’s less surprising than it could be for Skydio to come out of nowhere with this, because of where the company comes from. Skydio was founded by Bry and his labmate Abe Bachrach, from MIT’s Robust Robotics Group. Both Bry and Bachrach spent a year and a half at Google X (now just called X, following the company’s reorganization as Alphabet) working on Project Wing after graduating from MIT, but left to start Skydio in 2014. Google’s interest in these guys is no surprise; the research that they did at MIT culminated in a demonstration of a fixed-wing drone autonomously zipping around an underground parking garage:
The parking garage demo used a pre-existing map, but the video also shows projects that Bry and Bachrach worked on at MIT enabling dynamic on-board obstacle avoidance for quadrotors. It’s this research formed the foundation for Skydio, as Bry explains:
A lot of what we learned at MIT we’re incorporating into our products at Skydio. When you do a really good job on all the low level pieces—planning, control, 3D perception—the end behavior is magical. The airplane flying through the parking garage was an incredible demonstration, but it wasn’t something we could share with the world in any kind of scalable way. It relied on a pre-mapped environment and a heavy, expensive LIDAR sensor. At Skydio we’ve developed similar capabilities using sensors that weigh and cost almost nothing. Putting that kind of capability in consumers hands is an incredibly exciting thing to get to be a part of.
So how does this compare to the current state-of-the-art in consumer or delivery drones? Well, as far as we know, no delivery drones have demonstrated dynamic sense and avoid. But DJI did just release a new version of its Phantom drone, the Phantom 4, that includes vision-based sense and avoid out of the box:
Pretty cool, but the difference, Bry says, is that Skydio isn’t just avoiding obstacles: Skydio’s drone is “using all of the available information from the environment to make intelligent decisions to get smooth, intelligent behavior—similar to what an expert pilot would do.” In other words, Skydio’s hardware and software allows it to continually path-plan around obstacles with the sort of grace that robots (usually) don’t have. Essentially, it’s the difference between detecting and avoiding obstacles (not running into stuff) and doing dynamic mapping, navigation, and path planning. In the context of driving a car, that’s the difference between not crashing into something and actually getting where you want to go in an efficient and useful way.
Skydio’s drone is very obviously a prototype, but their technology has impressed enough people that they wrapped up a $25-million Series A back in January. They won’t tell us what exactly they’re working on (or even let us see a close-up picture of their drone), but Bry is optimistic about what Skydio will be able to accomplish:
Part of what makes this exciting is the long runway of stuff to be built and the new products and use-cases that will open up as the technology matures. Ultimately, all the information a drone needs to be good at its job is in the images. The challenge is extracting it and using it. Full scene understanding, deeper context awareness, learning based on user data and feedback, will all unlock more powerful autonomy. I think the ultimate test will be how these products perform in customer’s hands—our goal is to provide a trustworthy and magical experience.