Anyone who’s worked with a mobile manipulator will be bitterly familiar with grasping failures: you ask your robot to pick something up, and for whatever reason, it refuses to do it. Often, the whatever reason is because the robot can’t figure out how to properly position its gripper relative to the object, because the object is in a weird place that would lead to a gripper collision, like up against the side of a shelf.
A few years ago at IROS, Zhe Xu and Maya Cakmak at the University of Washington in Seattle presented a paper about how equipping mobile manipulators (like the PR2) with simple 3D printed tools can help them be much more effective at gripping (and consequently using) objects. Last month at ICRA, Cakmak presented another paper (with lead author Sarah Elliott) on a new kind of tool specifically designed to make it possible for robots to reorient and grasp objects that would otherwise not be graspable. The tool is “a rectangular prism” with a handle and “a textured silicon area with a high friction constant.” In other words, it’s a robot-friendly grippy poking stick.
We’ve seen similar work before where the robot uses its gripper to nudge objects in a confined or cluttered environment to make them easier to grasp, but while a gripper may be convenient for doing that, it’s not necessarily the best tool for the job. UW designed their grippy poking stick “to be ideal for pushing objects through point and surface contacts within confined spaces. At the same time, the high friction end-effector of the tool allows for pulling objects without requiring an articulated form that reaches behind the object.” That last thing, being able to use the end of the tool to pull objects toward you, enables a much wider array of actions.
Point cloud of scene as seen by the robot, with localized shelf (left) and detected object and tool (right).Image: UW
The grippy poking stick is mounted at an easily accessible point on the PR2’s shoulder, and the robot can grab it with a pre-specified motion whenever it needs to. PR2’s built-in grasping software is clever enough to tell whether or not it’ll be able to successfully grasp something, so the tricky bit is to get the robot to be able to say, “Okay, I can’t grasp this, but if I take my tool and poke at it in this way, the object will become graspable.” Rather than try to program that sort of thing in by hand, which sounds tedious and boring, the UW researchers instead got the robot to create its own predictive model of how tool actions will affect objects from learned experience.
As you might expect, it’s not trivial for a robot to model how objects will behave when poked and prodded, because their behavior depends on their physical properties, the physical properties of the surface that they’re on, and exactly where and how force is applied to them. Fortunately, the planning approach that UW developed allows the robot to use its tool on an object, and then plan a second action if necessary to bring the object closer to a graspable state. It can do this over and over again, although generally, it just takes one or two to make things work, and overall, the PR2 was able to demonstrate a successful rate of grasping. The researchers mention that besides just manipulating objects that are grasp targets, this tool and technique could also be applied to other objects around the target object: if the thing you want to grasp is buried on a shelf amidst a bunch of other stuff, your robot could nudge all of that clutter away to give itself room for a clean grasp.
Building and programming a home robot capable of manipulation is hard enough as it is, so a realistic interim approach might be to equip such robots with a cheap but effective belt of tools (including grippy poking sticks) that they can whip out as necessary. Or better yet, network the robot in with a 3D printer, allowing them to create what they need by themselves on demand, either choosing from a selection of handy pre-existing tools or (maybe, eventually) designing their own to be able to do exactly what they need.