An eye-catching dress that’s part art project, part cutting-edge tech, was presented today at the Ars Electronica Festival in Linz, Austria. The dress showcases an ultra-low energy, high resolution, brain-computer interface that’s so sensitive that if the wearer thinks about moving a single finger, it can identify which one—without the need for an implant.
The dress is the result of a collaboration between researchers at Johannes Kepler University Linz (JKU), developers at medical engineering company G.tec, and fashiontech designer Anouk Wipprecht. Known as the Pangolin dress, it has 1,024 individual head-mounted electrodes in 64 groups of 16. These detect electrical signals coming from the brain. The data from these sensors is combined, analyzed, and converted into colors displayed by 32 Neopixel LEDs and 32 servo-driven scales, creating a whole-body visualization of neural activity.
The biggest technical advance is in the new custom chip connected to each electrode. Measuring 1.6 millimeters on a side, the single-channel chip integrates an amplifier, an analog-to-digital converter (ADC), and a digital signal processor—and it draws less than 5 microwatts of power. Because it uses so little power, the chip can be powered by a nearby basestation in the manner of a contactless RFID chip, and return data wirelessly as well (although for the Pangolin dress, wired data connections were used to allow the creators to focus on developing and debugging other system elements while COVID-19 restrictions limited access to facilities.) Powering and communicating with the chip wirelessly would eliminate the need to tether a patient to a test system in a wired system, and even eliminate the bulk and weight of a batteries found in conventional wireless BCI systems. The biggest challenge was fitting into “the power budget for the sensor electronics,” says Harald Pretl, professor at the Institute for Integrated Circuits at JKU. “We had to create an amplifier design dedicated to this, an ADC dedicated to this, and also, based on ultrawideband, our own transmission protocol.” Another challenge was dealing with “fading and shadowing effects around the head,” adds Pretl.
But although the chip design was a custom build, it was deliberately made without recourse to exotic fabrication techniques: “We tried to focus on not using expensive technology there. We are using 180-nanometer [fabrication] technology, but we were able to get higher performance by using some advanced circuit design tricks,” explains JKU’s Thomas Faseth, who is the Pangolin dress’s project lead.
For the Pangolin dress, each group of electrodes and chips was mounted on a six-sided tile, and the wearer’s head—in this case, that of Megi Gin in Austria [below], who became as much a test participant as model—was covered in tiles, each hosting a custom chip and antenna.
Photos: Sarah Breinbauer
The next step up the technology stack was provided by Austrian commercial BCI developer and manufacturer, G.tec. G.tec brought its experience of analyzing neural signals to bear, integrating and interpreting the data coming from the tiles. For example, when you decide to move a muscle voluntarily, it sets off a localized pattern of activity in your motor cortex that can be detected and identified, “With 1,024 separate channels, you can get single finger resolution. This is something you normally cannot do with surface electrodes, you need implants,” says G.tec co-founder Christoph Guger. Normally, says Guger, surface electrode systems have 64 channels, which is only enough to distinguish, for example, whether the movement is intended for the right or left arm.
Because brains vary, identifying such fine grained detail requires calibrating the system to the individual wearer, using machine learning to recognize the patterns associated with different motions. However, the system doesn’t actually need you to be able to move any given muscle—in fact it learns more easily when the participant simply imagines performing a movement, because imagining a movement typically takes longer than performing it, producing a more sustained signal. This means that patients with paralyzed or missing limbs may be able to avail of the technology, and Guger even speculates as far as the possibility of BCI-controlled exoskeletons.
The Pangolin dress determines whether the wearer is one of a number of mental states, including stressed, neutral, and meditative, and expresses this through motion and light. The dress itself was created by designer Wipprecht, who previously wrote for IEEE Spectrum about her collaboration with G.tec in designing the Unicorn headset, intended to help therapists tune programs for children with ADHD.
The full length dress—this photo was taken in Florida, so it does not include the full set of head-mounted sensorsPhoto: Yanni de Melo
The dress combines rigid and fabric elements, with the rigid elements 3D-printed using selective laser sintering in nine interlocking segments. The modular nature of the sensors sparked the idea of taking inspiration from the flexible pangolin’s often lustrous keratin scales, says Wipprecht. For Wipprecht, part of the challenge was that brain signals can change much faster than a servo can change angle, so she decided to focus on reflecting the frequency of change instead: i.e when low frequency brain waves predominate, the servos and lights move and pulse slowly. "So much of the interaction design in my pieces is about creating an elegant behavior that gives a good sense of what the data is doing. In this case, since we have a lot of data points through all the 1,024 channels in the dress, my interactions can become even more spot-on … A hectic state is reflected in quick jittery movements and white lights, a neutral state is reflected in neutral movements and blue lights, and a meditative state is where the dress will turn purple with smooth, flowing movements,” says Wipprecht.
The Pangolin collaboration began late last year and continued despite the complication of Covid-19 restrictions. Although the JKU and G.tec teams were located in Austria, Wipprecht is in the hard-hit state of Florida. For the Austrian teams this meant severely restricted access to their labs, but they were able to send four sensor modules to Wipprecht as she worked on the dress design. The completed dress only arrived in Austria last month for final testing.
The Pangolin team hopes to create another iteration of the dress, this time using the sensors in a completely wireless mode. Discussions about commercializing the sensor technology are ongoing, but Guger estimates that within about two years it could be an off-the-shelf option for other researchers.
