The August 2022 issue of IEEE Spectrum is here!

Close bar

Dream Jobs 2011: Play It Again, Sergei

John Q. Walker brings back the sounds of musical greats

4 min read
man and piano
Photo: D.L. Anderson

dream jobs link box

John Q. Walker II

This profile is part of IEEE Spectrum's Special Report on Dream Jobs 2011.

When John Q. Walker was a teenager, his parents had him take piano lessons from one of their friends, a middle-aged woman who lived down the street.

That was a fortunate arrangement.

Ruth Slenczynska was no run-of-the-mill piano teacher. As a child prodigy, she had studied under Sergei Rachmaninoff, the famous Russian pianist, composer, and conductor, who died in 1943.

During lessons, the young Walker often heard Slenczynska say, "Now, that’s not how Mr. Rachmaninoff would have played it." That got him thinking: Could he figure out, in full quantitative detail, how Rachmaninoff would have played a piece? Decades later, Walker is still on that quest. And he’s having a blast.

Exploring his dual interests, he earned bachelor’s degrees in music and mathematics, and later he added a master’s degree in computer science, all from Southern Illinois University at Edwardsville, where Slenczynska was an artist in residence. He eventually received a doctorate, too, in software engineering, from the University of North Carolina at Chapel Hill. By that point, though, piano was more of a sideline than a possible vocation. "I was a natural who never practiced, so I was never very good," he says.

Walker spent 17 years working at IBM, principally in North Carolina’s Research Triangle Park, before starting a company called Ganymede Software with his colleague Peter Schwaller and two other engineers in 1995. Their software measured how fast computer networks pass data around. It was good timing: The following year the International Telecommunication Union began drawing up the first standards for voice communications over Internet Protocol. Because Ganymede’s software could help telecom providers determine how well their networks handled VoIP calls, the new company quickly took off. Its founders had equally good timing when they sold Ganymede, on 10 March 2000—the very day NASDAQ hit its peak.

Flush with cash and optimism, Walker could now pursue his teenage dream of reproducing the musical style of Rachmaninoff. Partnering with Schwaller again, he invested in a Yamaha Disklavier Pro, a robotic concert grand piano that can precisely play music fed to it in digital form. The two spent a year reverse engineering this exquisitely built but poorly documented machine. Then they turned to the even harder task of translating old musical recordings into digitized descriptions of exactly how each of the notes was played. Along the way, they hired a handful of other accomplished computer scientists with musical backgrounds for their new company, Zenph Sound Innovations.

To tackle that problem, they needed to transform music from raw audio into a digitized series of accurately described notes. Existing transcription software allowed a composer to transcribe noodlings at the keyboard, but it couldn’t capture all the nuances of a great piano performance—the careful timing of the sostenuto pedal, the measured touch of a quiet staccato note, the almost imperceptible pause between some musical phrases. That’s what the Zenph team set out to accomplish.

"We spent months and months on ’Mary Had a Little Lamb,’ " recalls Walker. Eventually they could encode digitally every detail of an extraordinarily complex piano performance: Glenn Gould’s 1955 recording of Bach’s Goldberg Variations. Although it wasn’t even recorded in stereo at the time, it remains one of the best-selling classical recordings ever.

In 2006 Walker and his colleagues arranged a meeting with the president of Sony Music Entertainment. They handed him a copy of the standard recording of Gould’s Goldberg Variations—to which Sony owns the rights—followed by their version of the same piece, done on the Yamaha Disklavier Pro using data that Zenph extracted from Gould’s original. It took the Sony president all of 3 minutes to go from "Who are these guys?" to "That’s fantastic—let’s do business," says Walker.

After clinching a deal with Sony, Zenph engineers continued to show off their technology by re-creating the playing of other famous musicians. "Reperformance" is what Walker calls the process. "We had to invent the word—so we did," he says.

Eventually, Walker says, he’ll have software to analyze many recordings and define the attributes that make one pianist different from another. At that point, it will be possible to create a virtual Gould or a virtual Rachmaninoff and to have a modern instrument play in their styles pieces these musicians never performed—a notion that might leave some music aficionados aghast but others delighted. "Here’s a different way to preserve an artist," Walker says, "in how they played."

And Walker isn’t going to stop with the piano. His team has already created reperformances of jazz bassists such as the late Ray Brown. Rather than attempting to do that robotically, Zenph synthesizes the sounds of a plucked bass using a very high-tech transducer that’s about as tall and wide as a string bass. "Now we’ve got Ray Brown in a can," quips Walker. He intends to work this way through a progression of instruments, ultimately even the human voice.

Zenph has other interesting services in mind as well. "You could upload to our Web site your daughter’s fourth-grade piano recital, and it would come back on a Steinway at Carnegie Hall," Walker says with a grin. The promise that computerized audio will soon do for music what computer graphics has already done for film gives Walker a huge charge, and the 54-year-old gushes with a teenager’s enthusiasm when he discusses the company’s future plans.

With its latest infusion of venture capital, Zenph recently opened a new office in Durham, N.C. Before that, most of the work was done around the kitchen table of Walker’s elegant Raleigh home, steps away from an 8- by 11-meter music studio he added to the house to hold Zenph’s high-tech instruments. He designed the space after one that the 19th-century composer Richard Wagner built at his home in Bayreuth, Germany. Walker’s former piano teacher, Slenczynska, occasionally helps with such tasks as choosing which concert grands to buy.

It’s obvious how much pleasure this high-tech business provides Walker. You can hear it in his laugh when he describes the marvelous things Zenph has done or is planning to do. So it’s clearly a rhetorical question when the musician and computer scientist asks, "This is a dream job—isn’t it?"

An abridged version of this article appeared in print as "Play It Again, Sergei."

To Probe Further

For more articles and special features, go to Dream Jobs 2011.

Keep reading...Show less

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions

Who Actually Owns Tesla’s Data?

The company, says the company—but other interpretations persist

4 min read
Nighttime photograph of a man in a car at an outdoor Tesla charging lot.

A Tesla user charges his Model S in Burbank, California.

Philip Cheung/The New York Times

On 29 September 2020, a masked man entered a branch of Wells Fargo bank in Washington, D.C., and handed the teller a note: “This is a robbery. Act calm give me all hundreds.” The teller complied. The man then fled the bank and jumped into a gray Tesla Model S. This was one of three bank robberies the man attempted the same day.

When FBI agents began investigating, they reviewed Washington, D.C.'s District Department of Transportation camera footage, and spotted a Tesla matching the getaway vehicle’s description. The license plate on that car showed that it was registered to Exelorate Enterprises LLC, the parent company of Steer EV—a DC-based monthly vehicle subscription service.

Keep Reading ↓Show less
{"imageShortcodeIds":[]}

Tesla’s Autopilot Depends on a Deluge of Data

But can a fire-hose approach solve self-driving’s biggest problems?

5 min read
Close-up of the Autopilot screen in a Tesla

In 2019, Elon Musk stood up at a Tesla day devoted to automated driving and said, “Essentially everyone’s training the network all the time, is what it amounts to. Whether Autopilot’s on or off, the network is being trained.”

Tesla’s suite of assistive and semi-autonomous technologies, collectively known as Autopilot, is among the most widely deployed—and undeniably the most controversial—driver-assistance systems on the road today. While many drivers love it, using it for a combined total of more than 5 billion kilometers, the technology has been involved in hundreds of crashes, some of them fatal, and is currently the subject of a comprehensive investigation by the National Highway Traffic Safety Administration.

Keep Reading ↓Show less

Get the Coursera Campus Skills Report 2022

Download the report to learn which job skills students need to build high-growth careers

1 min read

Get comprehensive insights into higher education skill trends based on data from 3.8M registered learners on Coursera, and learn clear steps you can take to ensure your institution's engineering curriculum is aligned with the needs of the current and future job market. Download the report now!