The February 2023 issue of IEEE Spectrum is here!

Close bar

Do it in Software or Hardware? VR Audio Takes on the Perennial Question

Dirac creates immersive audio for VR with algorithms while Harman patents flying robotic speakers

2 min read
Illustration of a woman looking up through a VR headset, with representations of audio, algorithms, and physical speakers around her.
Collage: IEEE Spectrum; Illustrations: iStockphoto

Hardware and software engineers, as the perennial lightbulb jokes point out, see the world differently. And indeed, a number of problems can be solved either way; the choice is not always obvious, at least at the time. The 737 Max design, reports are now indicating, turned out to involve a series of choices between solving problems in hardware or software.

Product designers are solving less critical problems all the time for which there are both hardware and software solutions. One of the more recent examples: how do you create realistic fully immersive audio for virtual reality that adjusts seamlessly as the user moves about the virtual world? Garden variety surround sound seems immersive, but generally doesn’t change as you move in relation to the speakers in a room, so the virtual experience can easily break apart.

Dirac, a company that develops digital signal processing techniques for optimizing audio, thinks the solution is software-centric. The company’s researchers measured what are called head-related transfer functions (HRTFs) for 30 people. HRTFs encode changes to a sound that occurs due to reflections and shadows caused by the shape of the head, ears, and torso. The researchers collected hundreds of HRTF measurements, instructing the subjects to move moved their heads around in relation to the rest of their bodies, and used these measurements to calculate a generic, dynamic, HRTF model. Virtual reality content products can use this model to make the sounds of a virtual environment respond to a user’s movements and appear to come from the virtual audio source, even when the actual environment contains just two speakers or headphones.

Researchers from Harman International in a recent patent application described an immersive audio system for virtual reality that relies on robotic speakers repositioning themselves as the virtual environment changes.Researchers from Harman International in a recent patent application described an immersive audio system for virtual reality that relies on robotic speakers repositioning themselves as the virtual environment changes.Image: U.S. Patent and Trademark Office

Meanwhile, audio equipment-maker Harman International (now part of Samsung) has sketched out a solution that requires lots of hardware. In a patent application for a mobile speaker system for virtual reality environments, Harman describes flying robotic speakers that reposition themselves as a user navigates a virtual world. The system includes a processor that maps the virtual environment to the user’s physical environment, “transmitting information to the robotic speaker to cause the robotic speaker to move to a location in a physical environment that corresponds to a location of the virtual object in the virtual environment, and transmitting an audio signal to the robotic speaker. The robotic speaker outputs, based on the audio signal and at the location in the physical environment, the sound associated with the virtual object.”

The Harman engineers have a number of ideas for how these robotic speakers would move around, including wheels, rotors, pulleys, and gas propulsion. If nothing else, such a system would certainly make watching someone navigate a virtual experience more entertaining.

The Conversation (0)

Asad Madni and the Life-Saving Sensor

His pivot from defense helped a tiny tuning-fork prevent SUV rollovers and plane crashes

11 min read
Asad Madni and the Life-Saving Sensor

In 1992, Asad M. Madni sat at the helm of BEI Sensors and Controls, overseeing a product line that included a variety of sensor and inertial-navigation devices, but its customers were less varied—mainly, the aerospace and defense electronics industries.

And he had a problem.

The Cold War had ended, crashing the U.S. defense industry. And business wasn’t going to come back anytime soon. BEI needed to identify and capture new customers—and quickly.

Keep Reading ↓Show less