Harmonix Music VR wants to change how you listen to music

“There’s nothing wrong with listening to music however you want to listen to music,” Jon Carter, the creative lead on PlayStation VR’s Harmonix Music VR, told Polygon.

“That said, there is a somewhat nostalgic image — a kid from some bygone era, sitting in their room and getting to know their music collection for hours on end — that serves as something of a driving inspiration for Harmonix Music VR,” he continued.

That’s one of the goals of the project. To get people to put some of their favorite songs on a USB stick, load up a visualizer, and bliss out to the music.

“It feels like an endangered activity,” Carter explained. “When you’re used to constant visual stimulation, like me and basically everyone I know, it can be physically difficult to engage with pure audio, unless it’s as an accompaniment to other activities. There’s nothing inherently wrong with this, but on the whole, I think our screen obsessions are not helping us appreciate all the great music out there with the same kind of depth and nuance that we used to.”

There’s a bit of irony is trying to fix our screen obsession with what amounts to one more screen, but you can’t argue with results. This can and does help you focus on the music and tune out the distractions of life.

This isn’t a game

Harmonix Music VR is a $14.99 program for PlayStation VR that offers four worlds with varying degrees of interactivity, and it works with any song you can put on a USB stick. There are experiences with interactions, but for me the real joy comes from simply loading up some good songs and enjoying the visuals. That’s it.

This either actively likely sounds either terrible or delightful, and the game is unlikely to convince you to change your gut reaction.

“It all depends on the attitude that people bring to the experience,” Carter said “Sometimes people quickly internalize what we’re going for with a ‘visualizer,’ but other times they sit down to try an experience from the people that made Rock Band, on hardware that they see as a gaming platform, and they’re at a loss as to why Music VR doesn’t give you any guided goals or player evaluation,” he continued.

The act of getting you inside this environment where your body and mind feel linked to the music, without distractions and without doing anything else, is tricky to communicate.

“People look around, sample all the interactions, and then feel ‘done,’” Carter explained. “I ended up having to prepare specific songs for specific playtesters, based on their music preferences, in order to stand a chance at getting them into the right frame of mind, where they’d see the music as the star, and all the fancy VR shenanigans as the supporting cast.”

How does this all work?

“There are two main sources of data we draw from for visualization: real-time, streaming audio spectrum analysis (“FFT”), and our own internally-developed ‘music information retrieval technology,” Carter said. “The former is what you see used in traditional visualizers, while the latter is our special sauce that gives us section detection, kick/snare detection, and even a value that represents the qualitative ‘energy’ of the music, among other things.”

The four worlds process and use this data in different ways, and a change in song can dramatically change the mood of each experience.

“For instance, on The Beach, our chill-est, introductory VR listening space, section detection drives weather patterns and atmospheric conditions, while the FFT data becomes prominent once you trigger one of the many gaze-activated visualizations,” he explained. “In The Easel, which is our psychedelic response to creation apps like Tiltbrush and Quill, the section and energy of the song determines the color palette of your creation, while FFT drives the psychedelic mutations on the brush strokes themselves. A lot of tuning went in to determining how each piece of audio data gets expressed in each world, so that no audio or video calibration is necessary for listener.”

You’ll see these processes work best in music with a 4/4 time, although anything you put into the system works. There are also 17 songs built into the program, although there’s no support for streaming services like Spotify.

Harmonix Music VR isn’t a game, but I had a surprisingly physical reaction to using the program to listen to my favorite songs. I wanted to move my body; I wanted to dance and even jump up and down. When you’re surrounded by a world that is reacting to the songs you’re listening to, it’s very easy to lose yourself in the music and reach a nearly ecstatic state — without the use of potentially harmful substances. Or perhaps in addition to, depending on your mood. There is even an experience called The Trip that is pure psychedelia, for when you just want to lean back and bliss out.

“Enthusiasts talk a big game about the disruptive potential for VR, but every time a new technology has broken through to the mainstream, it’s been by allowing for new, compelling ways of engaging with familiar activities,” Carter told Polygon. “Sure, there are a lot of gamers out there, but there are still far more music listeners.”

Harmonix Music VR will be released alongside the PlayStation VR hardware and is $14.99.