A Neuromorphic Chip That Makes Music

The prototype chip learns a style of music, then composes its own tunes

3 min read
This prototype chip learns a style of music, then composes its own tunes.
Photo: IMEC

A chip made by researchers at IMEC in Belgium uses brain-inspired circuits to compose melodies. The prototype neuromorphic chip learns the rules of musical composition by detecting patterns in the songs it’s exposed to. It then creates its own song in the same style. It’s an early demo from a  project to develop low-power, general purpose learning accelerators that could help tailor medical sensors to their wearers and enable personal electronics to learn their users’ patterns of behavior.

Today’s connected devices don’t have much smarts on board—instead they send data into the cloud for analysis by remote servers, where energy use and cooling costs are not at a premium, says Praveen Raghavan, who leads technology development for neuromorphic computation at IMEC. The IMEC team wants to change this. “The whole objective is to make artificial intelligence more compact, and bring it closer to the user,” he says. That means making compact, low-power dedicated learning chips. “We want to be as cost effective as possible,” he says.

The chip tune released by IMEC definitely sounds derivative. The composer has made a few odd note selections, but nothing that could be called avant garde—it’s in the mold of a certain strain of western classical music. Indeed, the 30-second tune evokes the simple melodies beginning musicians practice over and over. Two bars are very close to a riff on the chromatic scale; in the last bar, it resolves on a note that feels pleasing and expected.

For the chip to generate this tune, it was sequentially loaded with songs in the same time signature (which specifies how many beats are in each bar of music) and style. If exposed to a broad range of rhythms and styles, it wouldn’t have been able to discern the patterns at work. Raghavan says the prototype was taught using old Belgian and French flute minuets. Based on this, the chip learned the rules at play and then applied them. As the inputs are switched to a different time signature, it will start to learn that one, says Raghavan.

The prototype neuromorphic chip uses a one megabit bank of resistive RAM built on top of a processor. The memory cells are filaments of an oxide material whose electrical resistance varies along a continuum of states in response to different applied voltages. When the chip “hears” two particular notes one after the other in multiple training songs, it detects a pattern. The more often the notes occur together, the stronger the association between them, and the stronger the connection between the two memory cells that store them. In turn, it becomes more likely the chip will put those two notes in sequence when it switches from learning to composing mode. This is analogous to the way connections between different neurons in the brain strengthen or weaken as we learn. The system is hierarchical, and can spot and learn broader patterns, too. “It starts to see patterns in the music in the short term and long term,” says Raghavan.

Raghavan would not say how much power the prototype used—he says they overdesigned it so that it could take on a variety of tasks, building large memory cells that are responsive to a range of voltages. The music task doesn’t make use of the entire memory bank, either—but future, more complex tasks likely will. Now that they’ve studied the hardware’s capabilities, he says, they’ll optimize the design. Raghavan says it’s already flexible. The group has also trained the system with text, teaching it to autocomplete sentences.

Raghavan says these systems could be used as learning accelerators in personal electronics, or combined with other hardware designed to run neural networks. One application he has in mind is a heart-rate monitor for a smart watch that would be able to adjust to the user’s style to get more accurate readings. “Some people wear it tight, some loose, some higher or lower on the arm,” he says. All this can affect the accuracy of the measurements.

The Conversation (0)

Andrew Ng: Unbiggen AI

The AI pioneer says it’s time for smart-sized, “data-centric” solutions to big issues

10 min read
​Andrew Ng listens during the Power of Data: Sooner Than You Think global technology conference in Brooklyn, New York, on Wednesday, October 30, 2019.

Andrew Ng was involved in the rise of massive deep learning models trained on vast amounts of data, but now he’s preaching small-data solutions.

Cate Dingley/Bloomberg/Getty Images

Andrew Ng has serious street cred in artificial intelligence. He pioneered the use of graphics processing units (GPUs) to train deep learning models in the late 2000s with his students at Stanford University, cofounded Google Brain in 2011, and then served for three years as chief scientist for Baidu, where he helped build the Chinese tech giant’s AI group. So when he says he has identified the next big shift in artificial intelligence, people listen. And that’s what he told IEEE Spectrum in an exclusive Q&A.

Keep Reading ↓ Show less