The December 2022 issue of IEEE Spectrum is here!

Close bar

ETH Zurich Demonstrates PuppetMaster Robot

Robots that can control puppets could one day learn to manipulate complex physical objects like clothing and flexible sheets

2 min read
PuppetMaster Robot Pulls the Strings
While moving the puppet in real life, the robot is continuously simulating its next motions to try to get the puppet to move the way it's supposed to.
Image: ETH Zurich

As far as I know, the universe does not have a desperate need for robot puppeteers, and considering the difficulty of making even a halfway decent robot puppeteer, you’d think that any sensible roboticist would keep well clear of the problem. But some folks over at ETH Zurich decided that they’d have a crack at it anyway, and they started by describing why they’d likely be better off if they hadn’t:

Marionettes are underactuated, high-dimensional, highly non-linear coupled pendulum systems. They are driven by gravity, the tension forces generated by a small number of cables, and the internal forces arising from mechanical articulation constraints. As such, the map between the actions of a puppeteer and the motions performed by the marionette is notoriously unintuitive, and mastering this unique art form takes unfaltering dedication and a great deal of practice. Our goal is to enable autonomous robots to animate marionettes with a level of skill that approaches that of human puppeteers. 

I’m not much of a puppeteer myself, but this looks not bad at all, considering that the ABB YuMi robot is missing quite a few degrees of freedom in its hands. For context, here’s someone who has mastered this unique artform through unfaltering dedication and a great deal of practice, master puppeteer Scott Land: 

The ETH Zurich project can’t yet animate a complex marionette, but it’s a respectable showing with the dragon, I think. As input, all the robot needs to know is the design of the puppet at the target motion you want the puppet to make. While moving the puppet in real life, the robot is continuously simulating its motions over the next second while iteratively optimizing to try to get the puppet to move the way it’s supposed to.

The usefulness of this research, thankfully, is not constrained to puppets:

Our long term goal is to enable robots to manipulate various types of complex physical systems – clothing, soft parcels in warehouses or stores, flexible sheets and cables in hospitals or on construction sites, plush toys or bedding in our homes, etc – as skillfully as humans do. We believe the technical framework we have set up for robotic puppeteering will also prove useful in beginning to address this very important grand-challenge.

“PuppetMaster: Robotic Animation of Marionettes,” by Simon Zimmermann, Roi Poranne, James M. Bern, and Stelian Coros from ETH Zurich, was presented at Siggraph 2019.

[ Paper ]

The Conversation (0)

The Bionic-Hand Arms Race

The prosthetics industry is too focused on high-tech limbs that are complicated, costly, and often impractical

12 min read
A photograph of a young woman with brown eyes and neck length hair dyed rose gold sits at a white table. In one hand she holds a carbon fiber robotic arm and hand. Her other arm ends near her elbow. Her short sleeve shirt has a pattern on it of illustrated hands.

The author, Britt Young, holding her Ottobock bebionic bionic arm.

Gabriela Hasbun. Makeup: Maria Nguyen for MAC cosmetics; Hair: Joan Laqui for Living Proof

In Jules Verne’s 1865 novel From the Earth to the Moon, members of the fictitious Baltimore Gun Club, all disabled Civil War veterans, restlessly search for a new enemy to conquer. They had spent the war innovating new, deadlier weaponry. By the war’s end, with “not quite one arm between four persons, and exactly two legs between six,” these self-taught amputee-weaponsmiths decide to repurpose their skills toward a new projectile: a rocket ship.

The story of the Baltimore Gun Club propelling themselves to the moon is about the extraordinary masculine power of the veteran, who doesn’t simply “overcome” his disability; he derives power and ambition from it. Their “crutches, wooden legs, artificial arms, steel hooks, caoutchouc [rubber] jaws, silver craniums [and] platinum noses” don’t play leading roles in their personalities—they are merely tools on their bodies. These piecemeal men are unlikely crusaders of invention with an even more unlikely mission. And yet who better to design the next great leap in technology than men remade by technology themselves?

Keep Reading ↓Show less