A Better Way for Brains to Control Robotic Arms

A brain implant reads a paraplegic man’s intentions to let him pick up a beer

4 min read

A Better Way for Brains to Control Robotic Arms
Photo: Spencer Kellis & Christian Klaes/Caltech

/img/HRBMIErikSorto-1434464857340.jpgCheers! Erik Sorto uses a brain-controlled robotic arm to take a drink. “It gives me great pleasure to be part of the solution for improving paralyzed patients’ lives,” he says.Photo: Spencer Kellis & Christian Klaes/Caltech

Erik Sorto had been paralyzed for 10 years when hevolunteered for a bold neural engineering experiment: He would receive a brain implant and try to use the signals it recorded to control a robotic arm. Erik had no qualms about signing up for brain surgery, but his mother wasn’t happy about it. “She was just being a mom,” Sorto says with a smile. “She was like, ‘Your brain is the only part of your body that works just fine. Why would you mess with that?’ ”

Instead of reducing his capacities, the surgery gave Sorto superhuman abilities. In the experiments, Sorto simply imagines reaching out to grab an object and the robotic arm carries out his commands. While a handful of paralyzed people have previously used brain-computer interfaces (BCIs) to control robotic limbs, those subjects’ implants recorded signals from the primary motor cortex, which is linked directly to the spinal cord and muscles. Sorto’s implant was the first to record instead from the posterior parietal cortex, a brain region involved in planning movements.

human os icon

Lead researcher Richard Andersen, a neuroscience professor at Caltech, says that tapping into a subject’s intentions may provide more intuitive control of the robotic limb, and with less lag time. When the scientists asked Sorto to focus on the object he wished to grab, the system was able to identify that goal very quickly, says Andersen, usually within 200 milliseconds. The BCI then combined that high-level signal with smart robotics functions such as object recognition and simultaneous localization and mapping, or SLAM. “Erik worked with the robotic intelligence to manage the limb’s movement, rather than try to control all the kinematic details,” Andersen says. He and his colleagues revealed their novel approach recently in an article in Science.

To prepare for Sorto’s surgery in April 2013, the researchers first used functional magnetic resonance imaging to identify two precise regions of his parietal cortex that activated when he imagined reaching and grasping motions. The surgeons implanted two tiny microelectrode arrays, each with 96 electrodes that could record the electrical activity of single neurons. The grids linked to two metal “pedestals” that jutted out of Sorto’s skull. Within one month of surgery, Sorto was ready to get to work. The researchers connected cables to the pedestals, bringing the neural signals to a computer that analyzed them and sent commands to the robot arm. In the first experiment, a scientist made a simple twisting motion with his own wrist, and Sorto imagined himself imitating the gesture. And that’s just what the robot arm did. “It was pretty much effortless,” Sorto says.

From the population of neurons tapped by those electrodes, the researchers could distinguish cells whose activity coded for the location that Sorto wanted to reach, movement trajectories, and particular types of movement. Each day’s experiments began with system calibration, because the electrodes shifted slightly within Sorto’s head and picked up different neurons. “Our decoding algorithms took that into account,” Andersen says. If a given electrode was no longer contributing useful information to the decoding of a goal location, for example, the algorithm would ignore its signal and substitute other inputs. Andersen thinks such adaptive algorithms may enable a wide range of BCIs to record reliably over time.

Most of the prior studies in which paralyzed people used implanted BCIs were conducted by John Donoghue­, director of Brown University’s Institute for Brain Science and a pioneer in the use of implants in the motor cortex. Donoghue says that the new research advances our understanding of the parietal cortex’s role in generating movements and proves that it can provide a useful control signal. However, he’s not convinced it’s inherently a better signal than that provided by the motor cortex. “You get pretty good control from all these places, but it’s not clear how to get really good control,” he says.

While Andersen has suggested that combining the signals from the parietal and motor cortices might yield clearer commands for a robotic limb, Donoghue says that combination wouldn’t necessarily add value. “When you move your hand, you’re probably using 80 percent of your brain,” Donoghue says. The parietal and motor cortices provide similar signals, Donoghue says, so some other brain region may hold the key information that would result in truly natural movement. “It’s one of the great mysteries,” Donoghue says. “There’s something we still haven’t detected about the way the brain does movement.”

Sorto, meanwhile, has continued to work with what he has: More than two years after his surgery, his electrodes are still functioning and his enthusiasm is undimmed. In his second year of experiments, he mastered precise reaching and grasping movements by dint of an unusual exercise. “I played over 6,700 rounds of rock-paper-scissors,” Sorto says with agonized emphasis. “I want everybody to know that I worked hard.” It paid off, though, at the end of that second year, when he reached a long-held goal: He used the robot arm to lift a bottle of Modelo beer to his mouth and take a long swig.

This article originally appeared in print as “Telekinesis Made Simple.”

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions