With warehouses full of robots that can move shelves from place to place, the only reason that Amazon needs humans anymore is to pick things off of those shelves and put them into boxes, and pick other things out of boxes and put them onto those shelves. Amazon wants robots to be doing these tasks too, but it’s a hard problem—hard enough that the enormous bajillion dollar company is asking other roboticists to solve it for them.
The first Amazon Picking Challenge was held at ICRA last year in Seattle, Amazon’s home town. Amazon followed it up this year with another, tougher challenge at RoboCup 2016, which just wrapped up. And the winner is...Team Delft from the Netherlands! Relative to last year’s challenge, the performances were pretty impressive, and there’s already some video up on YouTube that shows how things went down.
This year’s Amazon Picking Challenge had two parts. From a “stationary and lightly populated inventory shelf” just like the ones that Kiva robots carry around, teams had to train their robot arms to autonomously do the following:
Stow Task — The stow task is to move 12 target items from a tote into bins on the shelf. The 12 target items in the tote will represent about 10 different products and be arranged so that some items are partially or completely occluded below other items. The tote will contain only the 12 target items. The target items can be stowed in any sequence and into any bin. The shelf will initially contain around 40 non-target items and each bin will contain between 1 and 10 items.
Pick Task — The pick task is to move 12 specified target items from the shelf into a tote. Each bin will contain between one and 10 items. Overall the 12 bins will contain around 50 items. There will be one target item for each bin and the target item must be picked from that bin. Some bins will feature a target item that is partially occluded or in contact with other items, but no items will be fully occluded. The shelf may contain all the announced items or a partial subset of them. The shelf may contain multiple copies of the same item, either in different bins or in the same bin. In the event there are multiple copies of a target item in a bin, any one can be picked, but not more than one. Target items may be picked in any sequence.
The actual items involved were selected “to represent popular kinds of products,” potentially including “books, cubic boxes, clothing, soft objects, and irregularly shaped objects.” Each robot had to do as much picking or stowing as it could in 15 minutes, and points were awarded for
stowing and picking each item, with more points going to items that were stowed in or picked from bins that were cluttered up with other stuff. Dropping things, or even nudging an item so that it stuck out of its bin more than 0.5 cm, resulted in penalties. And this is all totally autonomous, which the rules are unambiguous about: “no human interaction (remote or physical) is allowed with the Robot after uploading the work order and starting the robot.”
Got all that? Cool. Here’s what a pick looks like:
And here’s timelapse versions of winning team Delft rocking both the pick and stow tasks:
Notice how it’s able to deal with bins containing multiple items, both picking and stowing other items in those bins without knocking everything out onto the floor first. Impressive. The speed isn’t the greatest, but that’s something that can certainly be optimized. By way of comparison, humans can pick something like 400 items per hour at full speed, while Delft’s robot is currently clocking in at about 100 items per hour, with a failure rate of just over 16 percent.
At the moment, the easiest way of reliably picking up objects of various shapes and sizes and floppynessess without having to solve complicated grasping problems is with a suction gripper. However, the challenge very deliberately included a few items that were unsuitable for suction picking, including a wire trash can (no way to form a good seal) and a dumbbell (too heavy). Team Delft was prepared for this, and used two gripping systems: a suction gripper for most things, and then a more traditional pinch-grasp gripper for the other stuff:
Delft managed 214 points in the Stow task by cramming 11 items into densely packed bins, and 105 points in the Pick task, which was a tie with Team PFN from Japan (as a tie breaker, judges had to use video replay to determine which team had the fastest first pick). For winning both challenges, Delft gets $50,000
in Amazon gift cards. Interestingly, Amazon set a minimum score requirement for the full prize award of just 35 points per event. The fact that a dozen teams scored better than that suggests that this technology is progressing significantly faster Amazon may have expected.
Team Delft is a collaboration between Delft Robotics and TU Delft’s Robotics Institute, and we’re happy to report that the ROS-based software developed for the challenge will be released as open source.
Evan Ackerman is the senior writer for IEEE Spectrum's award-winning robotics blog, Automaton. Since 2007, he has written over 6,000 articles on robotics and emerging technology, covering conferences and events on every single continent except Antarctica (although he remains optimistic). In addition to Spectrum, Evan's work has appeared in a variety of other online publications including Gizmodo and Slate, and you may have heard him on NPR's Science Friday or the BBC World Service if you were listening at just the right time. Evan has an undergraduate degree in Martian geology, which he almost never gets to use, and still wants to be an astronaut when he grows up. In his spare time, he enjoys scuba diving, rehabilitating injured raptors, and playing bagpipes excellently.