“What did you enjoy the most about your trip to the Grand Canyon?” the Stanford researchers asked.
In response, a cursor floated across a computer screen displaying a keyboard and confidently picked out one letter at a time. The woman controlling the cursor didn’t have a mouse under her hand, though. She’s paralyzed due to amyotrophic lateral sclerosis (also called Lou Gehrig’s disease) and can’t move her hands. Instead, she steered the cursor using a chip implanted in her brain.
“I enjoyed the beauty,” she typed.
The woman was one of three participants in a study, published today in the journal eLife, that broke new ground in the use of brain-computer interfaces (BCIs) by people with paralysis. The woman who took the Grand Canyon trip demonstrated remarkable facility with a “free typing” task in which she answered questions however she chose. Another participant, a 64-year-old man paralyzed by a spinal cord injury, set a new record for speed in a “copy typing” task. Copying sentences like “The quick brown fox jumped over the lazy dog,” he typed at a relatively blistering rate of eight words per minute.
That’s four times as fast as the previous world’s best, says Stanford neurosurgeon Jaimie Henderson, a senior member of the research team. Further improvements to the user interface—including the kind of auto-complete software that’s standard on smartphones—should boost performance dramatically.
This experimental gear is far from being ready for clinical use: To send data from their implanted brain chips, the participants wear head-mounted components with wires that connect to the computer. But Henderson’s team, part of the multiuniversity BrainGate consortium, is contributing to the development of devices that can be used by people in their everyday lives, not just in the lab. “All our research is based on helping people with disabilities,” Henderson tells IEEE Spectrum.[shortcode ieee-pullquote quote=""When she became facile with this, she said it wasn't anything conscious; she felt like she was controlling a joystick."" float="right" expand=1]
Although getting brain surgery and having an implant installed is a drastic move, Henderson’s team recently surveyed people with paralysis regarding their willingness to use a variety of different BCI technologies. About 30 percent of respondents with injuries high on their spinal cords said they’d get a brain implant if it could be made wireless, and if it allowed them to type just two words per minute.
Here’s how the system works: The tiny implant, about the size of a baby aspirin, is inserted into the motor cortex, the part of the brain responsible for voluntary movement. The implant’s array of electrodes record electrical signals from neurons that “fire” as the person thinks of making a motion like moving their right hand—even if they’re paralyzed and can’t actually move it. The BrainGate decoding software interprets the signal and converts it into a command for the computer cursor.
Interestingly, the system worked best when the researchers customized it for each participant. To train the decoder, each person would imagine a series of different movements (like moving their whole right arm or wiggling their left thumb) while the researchers looked at the data coming from the electrodes and tried to find the most obvious and reliable signal.
Each participant ended up imagining a different movement to control the cursor. The woman with ALS imagined moving her index finger and thumb to control the cursor’s left-right and up-down motions. Henderson says that after a while, she didn’t have to think about moving the two digits independently. “When she became facile with this, she said it wasn’t anything conscious; she felt like she was controlling a joystick,” he says. The man with the spinal cord injury imagined moving his whole arm as if he were sliding a puck across a table. “Each participant settled on control modality that worked best,” Henderson says.
This experiment built on prior work by Henderson’s team involving both humans and monkeys. In a human study in 2015, the researchers tried out a new decoding algorithm that was better able to determine the user’s intended cursor direction. They demonstrated its effectiveness with a typing task that used a type of word-prediction software called Dasher, which enabled an ALS patient to type six words per minute.
Next came a primate study in 2016, which tested a further improvement to the algorithm: an ability to determine not just cursor direction but also understand when the user wanted to click on something. With this point-and-click system, the monkeys proved adept at moving the cursor to highlighted targets on the screen. By putting the highlighted targets on a display of letters, the researchers got the monkeys to mimic a typing task. Constructing sentences like Hamlet’s “To be or not to be—that is the question,” the monkeys achieved an impressive rate of 12 words per minute. But of course, they didn’t know what they were doing.
This latest study used the improved algorithms in the more naturalistic setting of a question-and-answer session. Study coauthor Paul Nuyujukian, director of Stanford’s new Brain Interfacing Laboratory, says they didn’t know what to expect from the free typing challenge.
“There was a piece of this that absolutely could not be answered until we got to a clinical study,” says Nuyujukian. “What happens when someone is synthesizing what they’re trying to communicate, and then communicating it in real time? That’s something we could not determine in the monkey lab.” It seemed possible that the mental act of deciding on an answer could interfere with the communication process, he says.
But the decoder proved up to the task, marking an important step toward a practical communication technology that people could use in their own homes.
Still, the challenges that remain are significant: BrainGate researchers want to make a system that’s fully implantable, wireless, and doesn’t require frequent recalibration to keep the decoder working properly. Henderson says investigators throughout the BrainGate group are now working on all those fronts.
Once the BCI gear is refined, it could be used to control other things besides a typing system. People with paralysis could use their brain signals to control wheelchairs, robotic arms, or even electrodes that stimulate their dormant muscles.
The Stanford researchers describe their work with the study’s three participants as a partnership, and say the three not only tested theories but also contributed to the technology’s development. In every testing session, say the researchers, the participants would give useful feedback about the system’s operation.
Nuyujukian recalls one occasion when the researchers were working with the woman who has ALS to test an altered algorithm for cursor control. “She said, ‘Something’s different, it doesn’t move right,’” Nuyujukian remembers. “After the second day of testing, we figured out that she was right; we had introduced a bug in the code.”
Eliza Strickland is a senior editor at IEEE Spectrum, where she covers AI, biomedical engineering, and other topics. She holds a master’s degree in journalism from Columbia University.