Ping Pong Robot Learns by Doing
Katharina Mülling (holding the emergency stop switch), Jan Peters, and Jens Kober monitor their ping pong robot practicing against a ball gun. All photos: Axel Griesch/MPG, München
Despite all the recent advances in robotics, one fundamental task appears to remain as hard as ever: robot programming.
To be sure, robot programming in industrial settings has evolved significantly, from a series of mechanical switches to advanced programming languages and teach-pendant devices for trajectory planning. But getting robots to do their jobs still requires a great deal of human labor -- and human intelligence.
The situation is even worse when it comes to programming robots to do things in non-industrial environments. Homes, offices, and hospitals are unstructured spaces, where robots need to deal with more uncertainty and act more safely.
To overcome this programming bottleneck, engineers need to create robots that are more flexible and adaptable -- robots that, like humans, learn by doing.
That's what a team led by Dr. Jan Peters at the Robot Learning Lab, part of the Max-Planck Institute for Biological Cybernetics, in Tübingen, Germany, is trying to do. Peters wants to transform robot programming into robot learning. In other words, he wants to design robots that can learn tasks effortlessly instead of requiring people to painstakingly determine their every move.
In the video below, you can see his students taking their robot "by the hand" to teach it motor skills needed for three tasks: paddle a ball on a string, play the ball-in-a-cup game, and hit a ping pong ball.
Here's how Dr. Peters explained to Automaton his team's approach: "Take the example of a person learning tennis. The teacher takes the student by the hand and shows basic movements: This is a forehand, this is a backhand, this is a serve. Still, it will take hours and hours of training before the student even feels comfortable at performing these behaviors. Even more practice is needed for the student to be able to play an actual game with these elementary behaviors." But still, he adds, humans succeed at learning the task. Why can't robots do the same? "That's what we're trying to do: Make our robots mimic the way humans learn new behaviors."
In the first part of the video, graduate student Katharina Muelling shows the robot how to paddle a ball on a string by performing the action while holding the robot's "hand." The robot decomposes the movement into primitive motor behaviors -- a discrete motor primitive that modulates the rhythmic paddling with an increasing amplitude until it becomes a stable rhythmic behavior -- and quickly "learns" how to perform the task.
For comparison purposes, the researchers tried to manually program the robot's motors to perform the same task. It took them three months and the result wasn't as good as the imitation learning experiment, which took less than an hour, Dr. Peters says.
In the second part of the video, Muelling teaches the robot the ball-in-a-cup game. [See photo on the right; the robot has to swing the yellow ball, which is attached to a string, and make it land into the blue cup.] This skill is significantly more difficult than paddling the ball on a string, and the robot doesn't have enough data to simply imitate what the human did. In fact, when the robot attempts to reproduce the human action, it can't match the accelerations of the human hand and the ball misses the cup by a large margin. Here, self-improvement becomes key, Dr. Peters says.
"For every new attempt, when the robot reduces the distance by which the ball misses the cup, the robot receives a 'reward,' " he says. "The robot subsequently self-improves on a trial-by-trial basis. It usually gets the ball in the cup for the first time after 40 to 45 trials and it succeeds all the time after about 90 to 95 trials."
How does the robot's learning ability compare to a human being? PhD student
Jens Kober, who led this particular experiment, wanted to find out: He went home for a holiday last year and enjoyed the benefit of an extended, large family -- always good subjects for a scientific experiment. He showed his many cousins the ball-in-a-cup game and rewarded them with chocolate. It turned out that the younger ones (around 6 years old) would not learn the behavior at all, the ones in their early teens (10 to 12) would learn it within 30 to 35 trials, and the grownups would be much faster.
"His supervisor may be the only person in his lab who has not managed to learn this task," Dr. Peters quips.
In the last part of the video, the researchers tackle an ever harder task: ping pong. Again, Muelling teaches the robot by holding its "hand," this time to hit a ping pong ball sent by a ball gun [photos above]. The challenge here is to use -- and modify -- previously learned basic motions and combine them with visual stimuli: The robot needs to keep track of the ball, which may come from different directions, and then execute the right set of motions.
Some of their work, part of GeRT consortium, a program that aims at generalizing robot manipulation tasks, is still preliminary, Dr. Peters notes. But he's confident they can teach their robot to become a good ping pong player. How good? Maybe not as good as Forrest Gump, but good enough to beat everyone in the lab.
Samuel Bouchard is a co-founder of Robotiq in Quebec City.