Jazz-Playing Robot Could Teach Us About Human-Computer Interaction
Benjamin Grosser, a professor of new media in the School of Art and Design at the University of Illinois at Urbana-Champaign, is developing a system that plays jazz improvisation in response to human jazz players.
Photo: L. Brian Stauffer

Researchers are developing a jazz-playing robot that listens to a human jazz musician and responds, in real time, by playing jazz improvisation.

“Humans tend to think that art is something that only humans can make,” says Benjamin Grosser, a professor of new media in the School of Art and Design at the University of Illinois at Urbana-Champaign. “I’m interested in what the boundaries of that are.”

In other words, is creativity only achievable by humans? Or, can computers also be artists?

“Computers don’t have a lifetime of cultural learning like we do,” he says. “So, they have a different vantage point of the world than we do.” In essence, this is what an artist is trying to accomplish.  

Because jazz improvisation is “cognitive but not linguistic,” says Grosser, this research could help gain insight into human-computer interaction. This might seem like a far-fetched concept, but we’ve already seen robots spray painting on walls and even drawing on the beach (one of Grosser’s previous projects was a robotic painting machine).

But, if you’re looking for a new jazz partner, you’re going to have to wait a bit. Researchers just started their five-year project, which is part of the new Communicating with Computers program sponsored by the U.S. Defense Advanced Research Projects Agency (DARPA).

According to Grosser, the front end of the computer system listens to the human player and analyzes the elements of what’s being played, including pitch, beat, and rhythm. This gets fed to the back end, an artificial intelligence system that’s tapped into a “knowledge base” of pre-analyzed jazz solos.

The system “analyzes incoming music the same way that it analyzes the jazz solos, using image schema as the mechanism,” he says. The real-time performance system then plays back jazz music that it “thinks” is within the boundary.

“By creating computational systems that act as artists, we not only start to reconsider what art is, who makes it, and what its boundaries are, but we also get a better look at how we make art and what we think as artists,” Grosser says. 

The Conversation (0)

The Bionic-Hand Arms Race

The prosthetics industry is too focused on high-tech limbs that are complicated, costly, and often impractical

12 min read
A photograph of a young woman with brown eyes and neck length hair dyed rose gold sits at a white table. In one hand she holds a carbon fiber robotic arm and hand. Her other arm ends near her elbow. Her short sleeve shirt has a pattern on it of illustrated hands.

The author, Britt Young, holding her Ottobock bebionic bionic arm.

Gabriela Hasbun. Makeup: Maria Nguyen for MAC cosmetics; Hair: Joan Laqui for Living Proof

In Jules Verne’s 1865 novel From the Earth to the Moon, members of the fictitious Baltimore Gun Club, all disabled Civil War veterans, restlessly search for a new enemy to conquer. They had spent the war innovating new, deadlier weaponry. By the war’s end, with “not quite one arm between four persons, and exactly two legs between six,” these self-taught amputee-weaponsmiths decide to repurpose their skills toward a new projectile: a rocket ship.

The story of the Baltimore Gun Club propelling themselves to the moon is about the extraordinary masculine power of the veteran, who doesn’t simply “overcome” his disability; he derives power and ambition from it. Their “crutches, wooden legs, artificial arms, steel hooks, caoutchouc [rubber] jaws, silver craniums [and] platinum noses” don’t play leading roles in their personalities—they are merely tools on their bodies. These piecemeal men are unlikely crusaders of invention with an even more unlikely mission. And yet who better to design the next great leap in technology than men remade by technology themselves?

Keep Reading ↓Show less