Google Cardboard adds 'inexpensive' spatial audio to its SDK

The conversation about virtual reality and the sense of presence — the ability for a program to feel like you're actually in another reality — usually centers on graphics.

"But an old game developer maxim is that it’s the sound that makes it real," Nathan Martz tells Polygon. Martz is a product manager working on the Google Cardboard team, with a focus on the software development kit. "If you want to create an immersive experience that runs on your phone, the best way to do that, the most cost-effective way to do that computationally is actually through audio," he explains.

Today the team is announcing the addition of spatial audio to the Cardboard SDK. When developers take advantage of the software they'll be able to create the illusion of three dimensional sound, even while the user is wearing standard headphones.

"An old game developer maxim is that it’s the sound that makes it real"

"The core idea is that in the real world we have two ears, and those ears are separated by distance but also by our brains and our skull," Martz said. "What happens in the real world is if you hear a sound to your left, you don’t just hear it in your left ear, you hear it in your left ear and your right ear with the sound in the right ear being delayed and it sounds a bit different. It hears sounds after they've passed through your skull. We can perform that kind of modeling in real time."

Modeling the user's physical head is just the first part of the software, the second step is modeling the environment around the user's head. "We have tools that allow developers to author this virtual sound environment," Martz explained. The sound waves would bounce around a cave much differently than they would out in the woods, and the software give developers control over how the sound interacts with the environment.

This had to be computationally cheap

Since Cardboard is a platform that is aimed at the mainstream — Cardboard games and experiences are playable on a wide variety of smart phones using a very inexpensive viewer — it was important that this software be as efficient as possible. "The SDK is optimized for mobile CPUs (e.g. SIMD instructions) and actually computes the audio in real-time on a separate thread, so most of the processing takes place outside of the primary CPU," the blog post states.

Developers are also able to control the degree of this effect on a per-sound basis. Do you need more fidelity for the voice of the character walking around the player, but don't mind if the dripping water effect in the background isn't quite as clear? You can make that happen.

"What I’m most proud of from this work is all the optimization our team has done to make these techniques practical on a mobile phone," Martz said. You can download an Android sample app to hear some of these effects in a virtual environment starting today, and the update to the SDK is available immediately.