Team Delft Wins Amazon Picking Challenge

Year two of the Amazon Picking Challenge results in robots that are much, much closer to taking over for humans

3 min read
Amazon Picking Challenge winner Team Delft robot arm
Photo: Team Delft

With warehouses full of robots that can move shelves from place to place, the only reason that Amazon needs humans anymore is to pick things off of those shelves and put them into boxes, and pick other things out of boxes and put them onto those shelves. Amazon wants robots to be doing these tasks too, but it’s a hard problem—hard enough that the enormous bajillion dollar company is asking other roboticists to solve it for them.

The first Amazon Picking Challenge was held at ICRA last year in Seattle, Amazon’s home town. Amazon followed it up this year with another, tougher challenge at RoboCup 2016, which just wrapped up. And the winner is...Team Delft from the Netherlands! Relative to last year’s challenge, the performances were pretty impressive, and there’s already some video up on YouTube that shows how things went down.

This year’s Amazon Picking Challenge had two parts. From a “stationary and lightly populated inventory shelf” just like the ones that Kiva robots carry around, teams had to train their robot arms to autonomously do the following:

Stow Task — The stow task is to move 12 target items from a tote into bins on the shelf. The 12 target items in the tote will represent about 10 different products and be arranged so that some items are partially or completely occluded below other items. The tote will contain only the 12 target items. The target items can be stowed in any sequence and into any bin. The shelf will initially contain around 40 non-target items and each bin will contain between 1 and 10 items.

Pick Task — The pick task is to move 12 specified target items from the shelf into a tote. Each bin will contain between one and 10 items. Overall the 12 bins will contain around 50 items. There will be one target item for each bin and the target item must be picked from that bin. Some bins will feature a target item that is partially occluded or in contact with other items, but no items will be fully occluded. The shelf may contain all the announced items or a partial subset of them. The shelf may contain multiple copies of the same item, either in different bins or in the same bin. In the event there are multiple copies of a target item in a bin, any one can be picked, but not more than one. Target items may be picked in any sequence.

The actual items involved were selected “to represent popular kinds of products,” potentially including “books, cubic boxes, clothing, soft objects, and irregularly shaped objects.” Each robot had to do as much picking or stowing as it could in 15 minutes, and points were awarded for 
stowing and picking each item, with more points going to items that were stowed in or picked from bins that were cluttered up with other stuff. Dropping things, or even nudging an item so that it stuck out of its bin more than 0.5 cm, resulted in penalties. And this is all totally autonomous, which the rules are unambiguous about: “no human interaction (remote or physical) is allowed with the Robot after uploading the work order and starting the robot.”

Got all that? Cool. Here’s what a pick looks like:

And here’s timelapse versions of winning team Delft rocking both the pick and stow tasks:

Notice how it’s able to deal with bins containing multiple items, both picking and stowing other items in those bins without knocking everything out onto the floor first. Impressive. The speed isn’t the greatest, but that’s something that can certainly be optimized. By way of comparison, humans can pick something like 400 items per hour at full speed, while Delft’s robot is currently clocking in at about 100 items per hour, with a failure rate of just over 16 percent.

At the moment, the easiest way of reliably picking up objects of various shapes and sizes and floppynessess without having to solve complicated grasping problems is with a suction gripper. However, the challenge very deliberately included a few items that were unsuitable for suction picking, including a wire trash can (no way to form a good seal) and a dumbbell (too heavy). Team Delft was prepared for this, and used two gripping systems: a suction gripper for most things, and then a more traditional pinch-grasp gripper for the other stuff:

Delft managed 214 points in the Stow task by cramming 11 items into densely packed bins, and 105 points in the Pick task, which was a tie with Team PFN from Japan (as a tie breaker, judges had to use video replay to determine which team had the fastest first pick). For winning both challenges, Delft gets $50,000 in Amazon gift cards. Interestingly, Amazon set a minimum score requirement for the full prize award of just 35 points per event. The fact that a dozen teams scored better than that suggests that this technology is progressing significantly faster Amazon may have expected.

Team Delft is a collaboration between Delft Robotics and TU Delft’s Robotics Institute, and we’re happy to report that the ROS-based software developed for the challenge will be released as open source. 

[ Amazon Picking Challenge ] via [ RoboValley ]

The Conversation (0)

How the U.S. Army Is Turning Robots Into Team Players

Engineers battle the limits of deep learning for battlefield bots

11 min read
Robot with threads near a fallen branch

RoMan, the Army Research Laboratory's robotic manipulator, considers the best way to grasp and move a tree branch at the Adelphi Laboratory Center, in Maryland.

Evan Ackerman

“I should probably not be standing this close," I think to myself, as the robot slowly approaches a large tree branch on the floor in front of me. It's not the size of the branch that makes me nervous—it's that the robot is operating autonomously, and that while I know what it's supposed to do, I'm not entirely sure what it will do. If everything works the way the roboticists at the U.S. Army Research Laboratory (ARL) in Adelphi, Md., expect, the robot will identify the branch, grasp it, and drag it out of the way. These folks know what they're doing, but I've spent enough time around robots that I take a small step backwards anyway.

This article is part of our special report on AI, “The Great AI Reckoning.”

The robot, named RoMan, for Robotic Manipulator, is about the size of a large lawn mower, with a tracked base that helps it handle most kinds of terrain. At the front, it has a squat torso equipped with cameras and depth sensors, as well as a pair of arms that were harvested from a prototype disaster-response robot originally developed at NASA's Jet Propulsion Laboratory for a DARPA robotics competition. RoMan's job today is roadway clearing, a multistep task that ARL wants the robot to complete as autonomously as possible. Instead of instructing the robot to grasp specific objects in specific ways and move them to specific places, the operators tell RoMan to "go clear a path." It's then up to the robot to make all the decisions necessary to achieve that objective.

Keep Reading ↓ Show less