The December 2022 issue of IEEE Spectrum is here!

Close bar

Finally: Robots Learn What 'Squishy' Really Means

Researchers are teaching robots to use common adjectives to describe how objects feel

2 min read
Finally: Robots Learn What 'Squishy' Really Means

Humans use all sorts of bizarre, abstract terms to describe how objects feel, and it’s endlessly frustrating to robots. Or at least, we imagine it must be. Take a word like "squishy," for example: how would you explain that feeling to a robot who experiences touch through some long series of numbers? Researchers at University of Pennsylvania's Haptics Group (part of the GRASP Lab) and UC Berkeley have developed a system to teach robots how these abstract terms apply to real-world objects, to help our mechanical friends communicate with us in a more relatable way.

In robotics, the science of touch and touch interaction is called haptics, and at the IEEE International Conference on Robotics and Automation (ICRA) this year, haptics has been all over the place. It’s a tricky thing to experiment with, because it requires sophisticated sensors, and equally sophisticated software to understand what the sensors are saying. Translating such sensor data into something that a human can understand is especially difficult, but in a paper presented this week, a PR2 robot equipped with an innovative finger sensor from SynTouch has been taught to use touch exploration to associate objects with "tactile adjectives."

A tactile adjective is a word like "squishy." "Fuzzy" is another one, and so is "crinkly." Humans easily understand what those terms mean. But robots still have a lot to learn. The study had a bunch of humans feel up a set of common household items, resulting in 34 adjective labels:

PR2 robot learns haptic adjectives.

The researchers then had a PR2 with the BioTac tactile finger sensor perform a series of exploratory procedures on the same set of objects, including tapping, squeezing, holding, and both slow and fast sliding. Here’s a video of PR2 exploring a folded satin pillowcase through touch:

After training the PR2 by correlating haptic sensor data with adjectives from humans who touched the same objects, the robot was tested out on a series of objects that it had never experienced before to see whether it would be able to derive the same haptic adjectives as humans do. And it worked. As shown in the video above (although it flashes past pretty quickly at the end), humans described the folded satin pillowcase as "compact, compressible, deformable, smooth, and squishy," while the robot thought it was "compact, compressible, crinkly, smooth, and squishy." I’m not sure where "crinkly" came from, but the rest of it is pretty close, and it’s very impressive for words that have a tendency towards subjectiveness. The researchers summarize:

The presented results prove that a robot equipped with rich multi-channel tactile sensors can discover the haptic properties of objects through physical interaction and then generalize this understanding across previously unfelt objects. Furthermore, we have shown that these object properties can be related to subjective human labels in the form of haptic adjectives, a task that has rarely been explored in the literature, though it stands to benefit a wide range of future applications in robotics.

"Using Robotic Exploratory Procedures to Learn the Meaning of Haptic Adjectives," by Vivian Chu, Ian McMahon, Lorenzo Riano, Craig G. McDonald, Qin He, Jorge Martinez Perez-Tejada, Michael Arrigo, Naomi Fitter, John C. Nappo, Trevor Darrell, and Katherine J. Kuchenbecker from the University of Pennsylvania and the University of California, Berkeley, was presented last week at ICRA 2013 in Germany.

The Conversation (0)

The Bionic-Hand Arms Race

The prosthetics industry is too focused on high-tech limbs that are complicated, costly, and often impractical

12 min read
A photograph of a young woman with brown eyes and neck length hair dyed rose gold sits at a white table. In one hand she holds a carbon fiber robotic arm and hand. Her other arm ends near her elbow. Her short sleeve shirt has a pattern on it of illustrated hands.

The author, Britt Young, holding her Ottobock bebionic bionic arm.

Gabriela Hasbun. Makeup: Maria Nguyen for MAC cosmetics; Hair: Joan Laqui for Living Proof

In Jules Verne’s 1865 novel From the Earth to the Moon, members of the fictitious Baltimore Gun Club, all disabled Civil War veterans, restlessly search for a new enemy to conquer. They had spent the war innovating new, deadlier weaponry. By the war’s end, with “not quite one arm between four persons, and exactly two legs between six,” these self-taught amputee-weaponsmiths decide to repurpose their skills toward a new projectile: a rocket ship.

The story of the Baltimore Gun Club propelling themselves to the moon is about the extraordinary masculine power of the veteran, who doesn’t simply “overcome” his disability; he derives power and ambition from it. Their “crutches, wooden legs, artificial arms, steel hooks, caoutchouc [rubber] jaws, silver craniums [and] platinum noses” don’t play leading roles in their personalities—they are merely tools on their bodies. These piecemeal men are unlikely crusaders of invention with an even more unlikely mission. And yet who better to design the next great leap in technology than men remade by technology themselves?

Keep Reading ↓Show less