One user played Beethoven’s “Ode to Joy” on an Android tablet piano app and later bought some groceries online. Another sent a few texts and then checked the weather forecast. A third browsed through some videos before firing up Stevie Nicks on Pandora.
They didn’t use their fingers to type commands or their voices to navigate the interface.
They used their noggins, specifically the motor cortex region of their brains where a baby aspirin-size chip had been implanted as part of a new study.
Last year, we highlighted a groundbreaking study from the BrainGate consortium in which the same brain-computer interface (BCI) enabled a paralyzed man to type up to eight words per minute via thoughts alone. In these new results from the BrainGate2 clinical trial, published today in the journal PLOS One, three participants with paralysis used the BCI to operate an off-the-shelf tablet.
“We wanted to see if we could allow somebody to control not an augmentative or alternative communication device, but the same ubiquitous device that people without physical disabilities use every day,” says study co-author Leigh Hochberg of Brown University, Massachusetts General Hospital and the Providence VA Medical Center.
All three participants suffer from weakness or loss of movement in their arms due to amyotrophic lateral sclerosis (ALS, also called Lou Gehrig’s disease) or spinal cord injury. Each received the brain implant, an array of microelectrodes, as part of the BrainGate2 clinical trial.
For this particular study, decoded neural signals from the implant were routed through an industry-standard Human Interface Device protocol, providing a virtual mouse. That “mouse” was paired to a Google Nexus 9 tablet via Bluetooth.
Each participant was asked to try out seven common apps on the tablet: email, chat, web browser, video sharing, music streaming, a weather program and a news aggregator. The researchers also asked the users if they wanted any additional apps, and subsequently added the keyboard app, grocery shopping on Amazon, and a calculator.
A BrainGate trial participant who is a musician played a snippet of 'Ode to Joy' on a digital piano interface.Photo: BrainGate Collaboration
The participants made up to 22 point-and-click selections per minute and typed up to 30 characters per minute in email and text programs. What’s more, all three participants really enjoyed using the tablet, says Hochberg.
As described in the paper, one user noted, “[T]he tablet became second nature to me, very intuitive. It felt more natural than the times I remember using a mouse.” Another said, “[A]mazing! I have more control over this than what I normally use.” The third added that he “loved” sending text messages via the tablet.
The BrainGate2 clinical trial is ongoing with 13 participants in prior and current studies, and the team continues to improve the speed and reliability of their neural decoding software, says Hochberg. The multiuniversity consortium is also working toward a wireless and fully implantable device with a rechargeable battery, similar to pacemakers and cochlear implants. “I’m optimistic that this technology is well on its way to following a similar path” as those technologies, he adds.
For users with paralysis, the ability to more freely communicate may be worth having surgery and getting a brain implant. Other research teams, most visibly Facebook, are working to develop non-invasive technologies that would translate “intended speech” into text without a brain implant, but those efforts have yet to show results.
“From our standpoint—and we’re fully dedicated and focused on helping people with paralysis and severe speech impairment—so far it feels like having a fully implanted device is going to provide a rapid and reliable method,” says Hochberg.
And with Thanksgiving in mind, he adds, “I’m incredibly grateful to not only the participants in our ongoing BrainGate trials, but everyone who participates in early clinical trials just to help others.”
Updated 21 Nov 2018
