Do it in Software or Hardware? VR Audio Takes on the Perennial Question

Dirac creates immersive audio for VR with algorithms while Harman patents flying robotic speakers

Illustration of a woman looking up through a VR headset, with representations of audio, algorithms, and physical speakers around her.
Collage: IEEE Spectrum; Illustrations: iStockphoto

Hardware and software engineers, as the perennial lightbulb jokes point out, see the world differently. And indeed, a number of problems can be solved either way; the choice is not always obvious, at least at the time. The 737 Max design, reports are now indicating, turned out to involve a series of choices between solving problems in hardware or software.

Product designers are solving less critical problems all the time for which there are both hardware and software solutions. One of the more recent examples: how do you create realistic fully immersive audio for virtual reality that adjusts seamlessly as the user moves about the virtual world? Garden variety surround sound seems immersive, but generally doesn’t change as you move in relation to the speakers in a room, so the virtual experience can easily break apart.

Dirac, a company that develops digital signal processing techniques for optimizing audio, thinks the solution is software-centric. The company’s researchers measured what are called head-related transfer functions (HRTFs) for 30 people. HRTFs encode changes to a sound that occurs due to reflections and shadows caused by the shape of the head, ears, and torso. The researchers collected hundreds of HRTF measurements, instructing the subjects to move moved their heads around in relation to the rest of their bodies, and used these measurements to calculate a generic, dynamic, HRTF model. Virtual reality content products can use this model to make the sounds of a virtual environment respond to a user’s movements and appear to come from the virtual audio source, even when the actual environment contains just two speakers or headphones.

Meanwhile, audio equipment-maker Harman International (now part of Samsung) has sketched out a solution that requires lots of hardware. In a patent application for a mobile speaker system for virtual reality environments, Harman describes flying robotic speakers that reposition themselves as a user navigates a virtual world. The system includes a processor that maps the virtual environment to the user’s physical environment, “transmitting information to the robotic speaker to cause the robotic speaker to move to a location in a physical environment that corresponds to a location of the virtual object in the virtual environment, and transmitting an audio signal to the robotic speaker. The robotic speaker outputs, based on the audio signal and at the location in the physical environment, the sound associated with the virtual object.”

The Harman engineers have a number of ideas for how these robotic speakers would move around, including wheels, rotors, pulleys, and gas propulsion. If nothing else, such a system would certainly make watching someone navigate a virtual experience more entertaining.

About View From the Valley blog

IEEE Spectrum’s blog featuring the people, places, and passions of the world of technologists in Silicon Valley and its environs.