Virtual Jamming

Four fixes for the major problem facing musicians who play together on the Web: lag

4 min read

You live in San Francisco and play the guitar, your bassist lives in Reno, and your drummer lives in Vancouver. To rehearse or even just get together and jam from so far away you need a way to meet in the same virtual performance room. The main problem is lag, the delay between the sounding of a note at one end of a circuit and its reception at the other end. Lag ”should be much less than during a telephone conversation,” says Stan Vonog, a Ph.D. candidate in physics at the Moscow Institute of Physics and Technology. You can manage to talk around a lag as high as 200 to 300 milliseconds, he says, but for multiplayer jam sessions, anything above 50ms renders a piece of music practically unplayable.

Vonog founded Musigy (pronounced MUSE-a-gee), based in San Francisco, to cut lag down to a manageable size. Just what's manageable depends, however, on how far apart the band members are. If the speed of light were the only constraint, you could keep lag under 50 ms across 8000 kilometers. Of course, electricity doesn't move quite that fast, and there are additional delays during signal processing and network rerouting. Together they bring the actual range down to 3200 km.

Musigy (https://musigy.com) fights lag in four ways. First, it shunts the audio signal directly from a PC's network card to its audio card, to keep the signal from waiting in the microprocessor's queue. Second, it encodes and decodes the audio signal with a speedy algorithm that adds just a 10-ms delay, a tenth of what a standard MP3 algorithm might add. Third, when a packet containing a bit of the audio signal doesn't arrive within a set period, say 40 ms, Musigy just leaves it out and extrapolates the missing data from earlier packets. Fourth, Musigy conducts a kind of ”weather forecast” when it sets up a jam session: the program tests standard lag times between the performers' stations and optimizes the settings accordingly.

In 2005 these ideas won Vonog and Musigy's other developers the Imagine Cup, a programming competition for students, sponsored by Microsoft.

To show off the product, Vonog sets up jam sessions. In September, for instance, he sponsored ”Jazz @ the Speed of Light,” which connected jazz, blues, and world-music players across some 2600 km, performing in real time in Russia, Ukraine, and the United Kingdom.

Right now, Musigy just comes in an invitation-only beta version. Vonog says that the commercial version will probably offer a two-tier pricing structure. ”Basic functionality like jamming or recording will be free,” he says. ”But if you want to upload all your rehearsals or hold live concerts, there will be a premium charge.”

Another tack, implemented by a company called eJamming (https://ejamming.com), is to cut lag by compressing data. That way, instead of swapping megabyte-size sound files, you could use mere kilobytes to describe when and what kind of notes each instrument plays. ”The analogy is a player piano with the small holes in the piano roll, which trigger the player piano to make sound,” says eJamming founder Alan Jay Glueckman. His compression standard is the familiar MIDI (Musical Instrument Digital Interface).

The company, based in Valley Village, Calif., and Orlando, Fla., now offers eJamming on a monthly subscription basis. Subscribers can sign on to the eJamming Web site and play with any other member who is connected. Glueckman cautions, however, that lag times begin to become noticeable at distances between about 320 km and 800 km.

”Some people feel it, some people don't,” he says. ”Some people will never be able to do more than 15 or 20 [ms].”

In March, eJamming announced its new AUDiiO package, which opens eJamming sessions up to as many as 16 low-bandwidth MIDI tracks and eight regular (high-bandwidth) audio tracks, such as guitar or vocals. Glueckman has also expressed interest in bringing eJamming AUDiiO to Second Life, the massive multi player environment where people meet via gamelike avatars.

Another approach to lag is to embrace it. Ninjam, a free, open-source program, does so by intentionally lagging each participant exactly one bar, much as choirs do when they sing rounds of ”Row, Row, Row Your Boat.”

Ninjam (https://ninjam.com) was conceived in large part by Justin Frankel, one of the creators of the peer-to-peer file- sharing program Gnutella and the audio player Winamp. ”Sometimes when you're playing in person, you don't always pay really close attention to what other people are doing,” he says. ”So how would you know if you're playing with their current measure or their previous measure?”

PHOTO: Alexander Gronsky/agency.photographer.ru

Look Ma, No Lag: Musigy nips delays in the bud in four ways.

The Ninjam session leader programs in the number of beats-per-minute and beats-per-measure in advance, the way you might plug a starting balance into a new Quicken account. Each participant then hears a metronome-click track to ensure everyone's on beat and an audio stream that Ninjam says will sound different to every user.

”This is obviously more suited to free-form experimentation than to rehearsing songs, because when you actually have changes it's very difficult to synchronize," says Frankel.

Ninjam is already available on Second Life. Dan Sullivan (avatar name, Jazz Glineux) holds Ninjam sessions in Second Life at his private performance space and recognizes the earmarks of a classic rivalry between Ninjam and eJamming users. ”Ninjam users tend to be skeptical about eJamming,” Sullivan says. ”It's a bit of the Microsoft pay model versus the free Linux thing.”

He says that just 10 ms of lag time is enough to notice and even trip over. He therefore remains skeptical about using Musigy, eJamming, and eJamming AUDiiO beyond a limited geographical range.

”Ten milliseconds,” he said, ”is about 900 miles, assuming an ideal point-to-point optical link between two players, from what I figure. And the Internet ain't no perfect connection.”

About the Author

MARK ANDERSON is an author and freelance writer based in Northampton, Mass. His work has appeared in Wired, Harper's, Discover, and Rolling Stone.

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions