The December 2022 issue of IEEE Spectrum is here!

Close bar

IROS 2013: Robots That Pay Attention To You

Now we can get robots to do stuff the same way that we get people to do stuff: by making eye contact, yelling, and waving

2 min read
IROS 2013: Robots That Pay Attention To You

Having a robot obey your commands is hard. Having a group of robots obey your commands is really, really hard. The good news is roboticists are making good progress in solving this problem. Research presented at IROS early this month shows how we may be getting to the point where teams of robots can pay attention to what we’re saying, where we’re looking, and what we’re doing, leading to a much more natural way to control them.

Think for a second about how you’d direct a subset of a group of people to do something. You’d probably look at each person in turn, saying something like “you, you, and you” while perhaps pointing to emphasize what you wanted them to do. It’s an intuitive thing for humans, because we convey attention through our gaze, and researchers from Simon Fraser University (which, incidentally, has an awesome bagpipe band) are teaching robots to pay attention to what we pay attention to.

Here’s a video illustrating the concept:

The AR Drones are watching the human with their cameras, and cooperatively using face detection to decide which one of them the human is looking at while listening for specific audio commands. Based on where the human’s attention is focused when verbal commands are issued, the robots can then decide which one of them the human is directing. It works if you want to select multiple robots (one at a time), or single robots, whether or not they’re already in the group.

The photos below show the camera views for the three robots. The robot that the user is staring at has the highest "face score" (left image).

The user gets feedback from the robots by watching their LEDs, which change color when the robot thinks it’s being paid attention to.

This same basic principle is just as effective with gestures:

The point of all this is to develop an intuitive command system that even an untrained user can quickly pick up on. To that end, future work will allow robots to be grouped into named teams, so that it’ll be possible to easily direct specific robots to join or abandon different groups. And gesture control will be expanded to allow for directional control. Put it all together, and you can imagine a scenario where you might be able to say to a group of six quadrotors: “you and you are Blue Team, go fly over there. You three are Red Team, patrol this area. Oh, and you, you’d better go join Blue Team.” It sounds so easy to do, and thanks to some very hard work by roboticists, we're getting there.

“ ‘You Two! Take Off!’: Creating, Modifying, and Commanding Groups of Robots Using Face Engagement and Indirect Speech in Voice Commands,” by Shokoofeh Pourmehr, Valiallah (Mani) Monajjemi, Richard Vaughan, and Greg Mori, and "HRI in the Sky: Creating and Commanding Teams of UAVs with a Vision-mediated Gestural Interface," by Valiallah (Mani) Monajjemi, Jens Wawerla, Richard Vaughan, and Greg Mori, all from Simon Fraser University in Burnaby, B.C., Canada., were presented at IROS 2013 in Tokyo, Japan.

The Conversation (0)

The Bionic-Hand Arms Race

The prosthetics industry is too focused on high-tech limbs that are complicated, costly, and often impractical

12 min read
A photograph of a young woman with brown eyes and neck length hair dyed rose gold sits at a white table. In one hand she holds a carbon fiber robotic arm and hand. Her other arm ends near her elbow. Her short sleeve shirt has a pattern on it of illustrated hands.

The author, Britt Young, holding her Ottobock bebionic bionic arm.

Gabriela Hasbun. Makeup: Maria Nguyen for MAC cosmetics; Hair: Joan Laqui for Living Proof

In Jules Verne’s 1865 novel From the Earth to the Moon, members of the fictitious Baltimore Gun Club, all disabled Civil War veterans, restlessly search for a new enemy to conquer. They had spent the war innovating new, deadlier weaponry. By the war’s end, with “not quite one arm between four persons, and exactly two legs between six,” these self-taught amputee-weaponsmiths decide to repurpose their skills toward a new projectile: a rocket ship.

The story of the Baltimore Gun Club propelling themselves to the moon is about the extraordinary masculine power of the veteran, who doesn’t simply “overcome” his disability; he derives power and ambition from it. Their “crutches, wooden legs, artificial arms, steel hooks, caoutchouc [rubber] jaws, silver craniums [and] platinum noses” don’t play leading roles in their personalities—they are merely tools on their bodies. These piecemeal men are unlikely crusaders of invention with an even more unlikely mission. And yet who better to design the next great leap in technology than men remade by technology themselves?

Keep Reading ↓Show less