The February 2023 issue of IEEE Spectrum is here!

Close bar

Inside an Experimental Robotics Class: A Robot Sketch Artist, a Robot That Plays Dominos, and More

Stanford’s Experimental Robotics class graduates eight very smart robots

3 min read
Inside an Experimental Robotics Class: A Robot Sketch Artist, a Robot That Plays Dominos, and More
Photo: Tekla Perry

Each year, Stanford Professor (and IEEE Fellow) Oussama Khatib introduces a new class of students to control theory and sets them loose on a room full of robot arms, including the Kuka LWR, the Kuka IIWA, the Barrett WAM, and the Kinova Jaco. The students in the Experimental Robotics class are charged with making the robots do something, typically, something that requires computer vision and force control. Said Khatib: each robot team had to develop “a strategy to draw or to play or to track, [because] the heart of robotics is perception connected to action.”

Typically, these industrial robot arms are programmed to perform factory tasks, like assembly, welding, and painting. The Stanford students get a rare opportunity to use them in more imaginative ways.

imgStanford Experimental Robotics students turned industrial robots to creative pursuitsPhoto: Tekla Perry

Many of the teams did far more than connect perception to action with their projects, assigning tasks to their robots that required a lot of smarts—like figuring out the best move in a game of dominos, or staying one step ahead of a moving model space station to prepare for docking, or learning to play a tune on a xylophone by listening to it.  [video below]

And this year’s teams of students did something that was a pleasant surprise for Khatib: They all achieved their goals—not just intermediate steps—and that isn’t always the case for this class. (It was pretty amazing to me as well. It’s not often that I go to a fast-paced demo event and all the demos work without a hitch.)

That’s not to say these robots couldn’t get better at the tasks in front of them with a little more time. (The students spent the first half of the 10-week class learning basic concepts; they had less than four weeks to actually build their projects.) But for me, it was their quirks that made them loveable. My favorite robot of the afternoon was “Sketch,” a robotic sketch artist [video, above]. Sketch can take any JPEG image and render it as a sketch. For the event, its creators gave it an apparently simple task—the Stanford logo (a block “S” with a tree in front of it). They ended up apologizing for how long the robot took (about eight minutes) as it dabbed in the leaves of the tree, one by one, and went back to fill in various parts of the S, a dot at a time. Said one team member: “As you can see, it is meticulously dotting the leaves on the trees. We have a bit of an OCD robot, but the good news is that at the end it creates very realistic images.” But, to me, it’s the inefficiency that made it seem much like a human artist.

The projects:

  • Picasso: Picasso’s team designed a robotic drawing system that can be operated remotely, wirelessly translating a sketch made by a tele-operator into strokes on a white board
  • CameraBot: CameraBot’s team aimed to develop a robotic cameraperson. The robot arm smoothly tracks a selected subject, getting its instructions from gestured commands. Arm gestures also allow one person to transfer control to another person
  • Xylophone: This robot listens to a tune played on a xylophone and reproduces it on the same instrument. One challenge, its developers pointed out, was to get the robot to tap the keys in a way that makes them resonate instead of clunk. [video, below]
  • Mind Control: The mind control robot uses alpha waves, sent from a helmet worn by the user, to determine where to direct a laser pointer.
  • Docking: Inspired by the movie Interstellar, Docking’s team made a model space station and developed a robot that uses vision based navigation to dock with it.
  • Force Control: Force Control’s team developed a method for allowing a robot arm to maintain constant contact against another object, even when the object makes sudden moves or changes in orientation.
  • DomiBot: DomiBot’s team developed a robot that plays dominos. It uses computer vision to see what’s on each domino, runs an algorithm to understand the position and orientation of each piece, considers the played and unplayed dominos it can see to determine its best move, and moves a selected domino into position.  In the demo, the robot played a game called Mexican Train against a human opponent; the two tied.
  • Sketch: The Sketch robot turns any JPEG image (including photographs) into a drawing. In the demo, it used just a single marker, however, the team showed an attachment they designed that allows the robot to easily change markers for a multicolored sketch.

The Conversation (0)

The Bionic-Hand Arms Race

The prosthetics industry is too focused on high-tech limbs that are complicated, costly, and often impractical

12 min read
A photograph of a young woman with brown eyes and neck length hair dyed rose gold sits at a white table. In one hand she holds a carbon fiber robotic arm and hand. Her other arm ends near her elbow. Her short sleeve shirt has a pattern on it of illustrated hands.

The author, Britt Young, holding her Ottobock bebionic bionic arm.

Gabriela Hasbun. Makeup: Maria Nguyen for MAC cosmetics; Hair: Joan Laqui for Living Proof

In Jules Verne’s 1865 novel From the Earth to the Moon, members of the fictitious Baltimore Gun Club, all disabled Civil War veterans, restlessly search for a new enemy to conquer. They had spent the war innovating new, deadlier weaponry. By the war’s end, with “not quite one arm between four persons, and exactly two legs between six,” these self-taught amputee-weaponsmiths decide to repurpose their skills toward a new projectile: a rocket ship.

The story of the Baltimore Gun Club propelling themselves to the moon is about the extraordinary masculine power of the veteran, who doesn’t simply “overcome” his disability; he derives power and ambition from it. Their “crutches, wooden legs, artificial arms, steel hooks, caoutchouc [rubber] jaws, silver craniums [and] platinum noses” don’t play leading roles in their personalities—they are merely tools on their bodies. These piecemeal men are unlikely crusaders of invention with an even more unlikely mission. And yet who better to design the next great leap in technology than men remade by technology themselves?

Keep Reading ↓Show less