Low-Cost Pliable Materials Transform Glove Into Sign-to-Text Machine

Smart glove features rubbery sensors, costs less than US $100, and converts sign language into text

2 min read
Photo: University of California San Diego
Photo: University of California San Diego

Researchers have made a low-cost smart glove that can translate the American Sign Language alphabet into text and send the messages via Bluetooth to a smartphone or computer. The glove can also be used to control a virtual hand.

While it could aid the deaf community, its developers say the smart glove could prove really valuable for virtual and augmented reality, remote surgery, and defense uses like controlling bomb-diffusing robots.

This isn’t the first gesture-tracking glove. There are companies pursuing similar devices that recognize gestures for computer control, à la the 2002 film Minority Report. Some researchers have also specifically developed gloves that convert sign language into text or audible speech.

What’s different about the new glove is its use of extremely low-cost, pliable materials, says developer Darren Lipomi, a nanoengineering professor at the University of California, San Diego. The total cost of the components in the system reported in the journal PLOS ONE cost less than US $100, Lipomi says. And unlike other gesture-recognizing gloves, which use MEMS sensors made of brittle materials, the soft stretchable materials in Lipomi’s glove should make it more robust.

The key components of the new glove are flexible strain sensors made of a rubbery polymer. Lipomi and his team make the sensors by cutting narrow strips from a super-thin film of the polymer and coating them with conductive carbon paint.

Then they use a stretchy glue to attach nine sensors on the knuckles of an athletic leather glove, two on each finger and one on the thumb. Thin, stainless steel threads connect each sensor to a circuit board attached at the wrist. The board also has an accelerometer and a Bluetooth transmitter.

When the wearer bends their fingers, the sensors stretch and the electrical resistance across them goes up. Based on these resistance signals, the circuit assigns a digital bit to each knuckle, 0 for relaxed and 1 for bent. This creates a nine-bit code for each hand gesture of the ASL alphabet. So if all fingers are straight, the code reads 000000000; for a fist it would be 111111111.

To distinguish between ASL letters that generate the same code, the researchers incorporated an accelerometer and pressure sensors on the glove. The letters D and Z, for instance, have the same gesture but the hand zigzags for Z while it remains still for D. In U and V, meanwhile, two fingers are held together and apart respectively, which the pressure sensor detects. 

In tests, the glove could translate all 26 letters of the American Sign Language alphabet into text. The research team also used the glove to control a virtual hand to sign the ASL letters.

The next version of the glove will incorporate new materials that generate a tactile response so that wearers can feel what they’re touching in virtual reality. Today’s haptic devices simulate the sense of touch by applying forces and vibrations to the user. Lipomi and his students plan to convey a much broader range of signals. “We’re synthesizing materials that can be used to stimulate everything from pressure and temperature to stickiness and sliminess,” he says.

The Conversation (0)

The Future of Deep Learning Is Photonic

Computing with light could slash the energy needs of neural networks

10 min read

This computer rendering depicts the pattern on a photonic chip that the author and his colleagues have devised for performing neural-network calculations using light.

Alexander Sludds
DarkBlue1

Think of the many tasks to which computers are being applied that in the not-so-distant past required human intuition. Computers routinely identify objects in images, transcribe speech, translate between languages, diagnose medical conditions, play complex games, and drive cars.

The technique that has empowered these stunning developments is called deep learning, a term that refers to mathematical models known as artificial neural networks. Deep learning is a subfield of machine learning, a branch of computer science based on fitting complex models to data.

Keep Reading ↓ Show less