Navigation by the Soles of Your Feet (and the Seat of Your Pants)

30 October 2009—The human gateway to the electronic world is mostly through our eyes and ears. But devices that connect through our sense of touch have been developed for the fingertips, the chest, the back, and even the tongue. Now researchers in Mexico have come up with a device to communicate via your feet, and a collaboration between a Dutch organization and General Motors has tested a way to communicate potentially vital driving information through none other than your rear end.

The point of these new tactile devices is to develop touch-sensitive applications for virtual reality, gaming, robotics, rehabilitation, navigation, and assistance for the blind or visually impaired, among others things.

The shoe dropped at the recent IEEE/RSJ International Conference on Intelligent Robots and Systems. Ramiro Velazquez, an assistant professor in the Mechatronics and Control Systems Lab at Mexico's Panamerican University, presented his computer-sole interface, which will be worn inside a shoe in its final version. It's the first device to stimulate the bottom of the feet to convey information, rather than to enhance sensation. The goal of his study, Velazquez says, was to answer the question, "Are we capable of understanding information through our feet?"

The answer, it turns out, is yes.

His group chose to stimulate an area of the foot that has a high concentration of receptors for texture and vibration sensing—around where the arch and the ball of the foot meet, along the outer edge of the sole. The researchers arrayed four rows of four miniature vibrating motors of the type used in cellphones, available commercially for US $10, in the shoe insert. Each of these 16 actuators could be activated independently and at different vibrating frequencies to transmit signals, meant to communicate directions and patterns, to 20 research subjects in their study.

First, the researchers tested whether people could understand "dynamic directions," or signals moving in certain directions on the bottom of the feet, while the subjects simply sat still. The researchers matched cardinal directions to patterns on the shoe insert by vibrating rows and columns of the actuators one after another. They vibrated heel row to toe row, for example, to indicate north, or forward, and reversed the direction from toe row to heel row to indicate south, or backward.

The researchers also tested the subjects' ability to perceive geometric shapes made by the vibrating actuators; patterns of vibration, like the alerts for calls or text messages on cellphones; and navigation cues using the dynamic directions as before but this time while the subjects were walking blindfolded around obstacles in a room. They found that people were best at sensing directions and recognizing patterns. In the navigation test, completed by five of the original 20 subjects, four tested well. One got pretty turned around, though even he eventually made it through the obstacle course.

Certain information is just not easily discernible when transmitted via foot communication, Velazquez found. The test subjects had difficulty determining geometric shapes, such as a line, a circle, or a square. Velazquez explains that because vibrations expand throughout the skin, very specific geometric information—such as a diagonal line—is difficult to distinguish.

Still, humans can glean a lot from the soles of their feet. Now that his group has seen what's possible, Velazquez hopes they can create "a new tactile language for the feet" that you'd learn just like any other language.

Carnegie Mellon Robotics Institute professor Mel Siegel says that although the device is "clearly an early prototype," it shows the promise of "really doing something useful," like aiding the visually impaired. It also comes at a time when our primary senses for navigation and mapping the world around us—our eyes and ears—are overloaded, Siegel points out. "Everyone talks about sensory overload," he says. But if you could use this "other modality"—touch—you might be able to take in important information without competition in the same arena from other visual or audio signals, Siegel says, although experiments would need to prove that hypothesis.

The car is a good example of a potentially overloaded sensory environment. Driver-assistance, crash-avoidance, communication, and entertainment systems all contribute to a crowd of audio and visual information knocking about in the vehicle. But what if we could take advantage of the underworked tactile sense?

That's a theory that researchers at General Motors and the Dutch research organization TNO took for a test-drive last year. Using a cushion that gives you directions straight through the seat of your pants, the researchers set out to discover if drivers could actually feel direction signals—not just in the lab but also in the harsh reality of driving on real roads, according to TNO researcher Jeroen Hogema. Would vibration signals be masked by rough roads? Would drivers fail to notice the signals if they were concentrating on the actual task of driving?

The researchers tested eight subjects on the device, which was "neatly designed into the upholstery," Hogema says, so it looked "like a normal seat of a car." The seat-cushion device had 64 motors, also of the "silent alarm" cellphone variety, and any of them could be actuated separately, although the researchers clustered groups of actuators to code for eight different directions: front, front left, front right, right, left, back, back left, and back right. They used a single type of test signal: three short, vibrating bursts. Subjects reported the signal as soon as they felt it and the direction as soon as they sensed it.

Like the sole-stimulating shoe, the vibrating seat cushion did its job. Researchers found that 93.3 percent of the responses were correct, while 6.4 percent were off by one direction segment and just 0.3 percent were off by two segments. In other words, Hogema explains, no one confused left with right or back with front. Rather, the mistakes were more subtle. There were "no situations where the chair gave a direction and the person didn't get the message at all," he says.

In addition to providing direction or navigation information, a rear-end-stimulating seat cushion could be used to avoid getting rear-ended. Seat-cushion direction cues might be an ideal output system for collision-avoidance radar. Although the researchers didn't test the car seat with that possibility in mind, they did show its potential by surprising their subjects with yet another message from the chair after the drivers thought the test was over. Every subject felt the unexpected signal quickly, and almost all of them picked the correct direction.

"It's encouraging that people can get the message" from such an unusual spot, Hogema says, and that they can "do so rapidly and with very few errors."

The next stage of this project would be to link the direction signals to a certain application, says Hogema. While his group at TNO is no longer involved, GM is still working on the project but is keeping mum on the details. TNO and GM are reporting their initial results in an upcoming issue of IEEE Transactions on Haptics.

Success with both prototypes—the shoe and the seat cushion—indicates that people can indeed use tactile cues to pick up directional information, although we're still a long way from navigating to the grocery store using the electronics on our feet or the cushion under our seat.

Still, even if these vibrators don't yield the right information, Velazquez jokes, they could provide excellent massages.