We live in a world of sounds, full of beautiful music, birds chirping, and the voices of our friends. It's a rich cacophony, with blaring beeps, accented alarms, and knock-knock jokes. The sound of a door opening can alert us to a friend's arrival, and a door slamming can alert us to an impending argument.

HEARBO (HEAR-ing roBOt) is a robot developed at Honda Research Institute–Japan (HRI-JP), and its job is to understand this world of sound, in a field called Computational Auditory Scene Analysis.

At the IEEE International Conference on Intelligent Robots and Systems and RO-MAN this year, several papers describing HEARBO's latest functionalities were introduced.

Super-hearing robots

With the dream of the futuristic robotic butler, researchers are trying to make robots understand our voice commands, a bit like Apple’s Siri but from 2 meters away. Typical approaches use a method called beamforming to "focus" on a sound, like a person speaking. The system then takes that sound, performs some noise reduction, and then tries to understand what the person is saying using automatic speech recognition.

The beamforming approach is widely used, but HEARBO takes the beamforming approach a step further. What about when the TV is on, the kids are playing on one side of the room, and the doorbell rings? Can our robot butler detect that? HEARBO researchers say it can, using their own 3-step paradigm: localization, separation, and recognition. This system, called HARK, lets you recover the original sounds from a mixture based on where the sounds are coming from. Their reasoning is that "noise" shouldn't just be suppressed, but be separated out and then analyzed afterwards, since the definition of noise is highly dependent on the situation. For example, a crying baby may be considered noise, or it may convey very important information.

At IROS 2012, Keisuke Nakamura of HRI-JP presented his new super-resolution sound source localization algorithm, which allows sounds to be detected to within 1-degree of accuracy. For example, it could precisely detect the location of a human calling for help in a disaster situation.

Using the methods developed by Kazuhiro Nakadai's team at HRI-JP, up to four different simultaneous sounds or voices can be detected and recognized in practice. Theoretically, with eight microphones, up to seven different sound sources can be separated and recognized at once, something that humans with two ears cannot do.

Understanding the mixture

Nakamura has also taught HEARBO about the concepts of music, human voice, and environmental sounds. For instance, HEARBO was trained with many different music songs to learn about the general characteristics of music. Upon hearing a song it’s never heard before, it can say, “I hear music!” This means that HEARBO can tell the difference between a human giving it commands, and a singer on the radio.

In this video, three sound sources are positioned around HEARBO: to the robot's right is a beeping alarm clock; in front of it, a speaker plays music; to its left a person speaks. The first thing the robot does is to capture all the sounds, recognize them, and determine the location they're coming from. Then HEARBO focuses its attention on each source one by one. Watch:

Ego-noise suppression

Robots also have some special robot-y hurdles to overcome. For example, when robots move, their motors whir and grind, distorting the sound they're trying to hear. Just like how our human hearing system filters out the sound of our own heartbeat, robots need to remove this self-generated noise. This is called ego-noise suppression and is the research focus of Gökhan Ince. How do they do it? They embed microphones in HEARBO's body, and subtract that internal noise in real-time from what the robot hears through its head mics. This robustness to noise is essential for helping robots decipher their world.

In this new dancing robot video developed in collaboration with INESC Porto and LIACC-FEUP, even when HEARBO is listening to the music, grooving its motors to the beat, it can still understand its human's commands. This robot is not just great dancer, but shows off the capabilities of a hearing robot adeptly handling a plethora of sounds, just like in real-life.

"Real-time Super-resolution Sound Source Localization for Robots" by Keisuke Nakamura, Kazuhiro Nakadai, and Gökhan Ince from Honda Research Institute, and "Live Assessment of Beat Tracking for Robot Audition" by Joao Lobato Oliveira, Gökhan Ince, Keisuke Nakamura, Kazuhiro Nakadai, Hiroshi G. Okuno, Luis Paulo Reis, and Fabien Gouyon from INESC Porto, LIACC-FEUP, HRI-JP, and Kyoto University were presented at IROS 2012 in Villamoura, Portugal.

[ HARK Robot Audition System ]

[ HARK Open-Source Code ]

The Conversation (0)

The Bionic-Hand Arms Race

The prosthetics industry is too focused on high-tech limbs that are complicated, costly, and often impractical

12 min read
A photograph of a young woman with brown eyes and neck length hair dyed rose gold sits at a white table. In one hand she holds a carbon fiber robotic arm and hand. Her other arm ends near her elbow. Her short sleeve shirt has a pattern on it of illustrated hands.

The author, Britt Young, holding her Ottobock bebionic bionic arm.

Gabriela Hasbun. Makeup: Maria Nguyen for MAC cosmetics; Hair: Joan Laqui for Living Proof

In Jules Verne’s 1865 novel From the Earth to the Moon, members of the fictitious Baltimore Gun Club, all disabled Civil War veterans, restlessly search for a new enemy to conquer. They had spent the war innovating new, deadlier weaponry. By the war’s end, with “not quite one arm between four persons, and exactly two legs between six,” these self-taught amputee-weaponsmiths decide to repurpose their skills toward a new projectile: a rocket ship.

The story of the Baltimore Gun Club propelling themselves to the moon is about the extraordinary masculine power of the veteran, who doesn’t simply “overcome” his disability; he derives power and ambition from it. Their “crutches, wooden legs, artificial arms, steel hooks, caoutchouc [rubber] jaws, silver craniums [and] platinum noses” don’t play leading roles in their personalities—they are merely tools on their bodies. These piecemeal men are unlikely crusaders of invention with an even more unlikely mission. And yet who better to design the next great leap in technology than men remade by technology themselves?

Keep Reading ↓Show less