VReal wants to be Twitch for VR

How do you explain virtual reality to someone who has never worn a VR headset? How do you relate a VR experience to someone who wasn’t there? And how do you extend the streaming experience, traditionally the domain of services like Twitch, YouTube and Facebook, into VR? VReal thinks it has the answer to these and other questions, and we got to try it out earlier this month.

The first requirement was to put on an HTC Vive headset, the consumer-grade VR system known for its partnership with Valve Software and having motion controllers as a standard. Next, I connected to VReal’s prototype streaming service, running on a private network at its headquarters in Seattle. Once inside, I was present in their virtual lobby, a space that resembled the open-air landing platforms you might see imagined in the Star Wars prequels or movies like Blade Runner.

Then, I clicked on a game and began watching.

That first game was called Cloudlands. We’ve written about it before, but think of it as an extremely dramatic interpretation of miniature golf. I wasn’t inhabiting the perspective of the player, though. Instead, I was simply present during their game, able to move around and see their avatar next to us while they played live.

And I wasn’t alone. There was another person there with me, in the shape of a portly panda bear.

Alright. Neat trick, I thought to myself. But VReal didn’t stop there.

Next, the demo team asked me to touch a button on the motion controller to spawn a camera into the game world. A little gray box appeared: a virtual camcorder. Using the Vive controllers, I could move it around with complete freedom and even zoom in and out. I could detach the virtual LCD screen, leaving it suspended in the air while I walked around the action holding the camera to get a better view.

Cloudlands

Todd Hooper, CEO and founder of VReal, says that these cameras can easily output straight to YouTube. There can be many virtual cameramen inside the game with players taking highlights from different angles, and all of those feeds can be chopped up and experienced with a traditional, two-dimensional display.

But the entire experience — my experience, from the perspective of my viewpoint — can be captured as well. VReal’s technology then re-renders the entire game live or after the fact, allowing viewers to re-live the experience inside a VR system of their choice.

VReal is, Hooper says, hardware-agnostic. It works with all the major systems. But rendering a scene in VR, at 60-90 frames per second, takes a lot of processing power. VReal seems to be doing twice the work. How is that possible with consumer hardware? Right now, Hooper isn’t saying. But he’s already started shopping for partners in the games industry who want to integrate the technology.

"We joke that there’s a lot of ways that you could try to do this technically," Hooper says. "The way that we have developed, it’s pretty straightforward to integrate. We’ve already signed a number of developers, which we’re going to be announcing later this year, but the goal is they can get up and running in a day.

VReal's virtual lobby.

"Unity and Unreal 4 are our target engines. I was previously part of the management team at Unity. I was there for two years. So that’s my bias, towards Unity. And I would say that 90 percent of the VR content that we’ve seen so far is Unity and Unreal 4. ... We can also render out to a 360-degree camera for GearVR ... Obviously it doesn’t have the hand-tracking controllers, but we think that there will be a lot of people that will want to watch streams on this device as well."

Next, his demo team took me into a mockup of a game, something it's calling "VR Legends." If you squinted really hard you could see it as a League of Legends clone, but it could also be a generic sort of real-time strategy game. And it’s here that the real fun began.

My Panda-shaped tour guide began to play with the game’s scale. There, alongside one of the lanes where blue minions are duking it out against red minions, is a stage. Pointing to it with my motion controller and hitting a button, I shrunk down to the same size as the minions on the map. I was on a stone platform, inside a canyon, looking up at its dusty virtual walls. And there, standing next to me was my guide the panda bear.

A representation of a player, looking down on the action, while a spectator watches below.

Looming above us was the person actually playing VR Legends, live in the other room next door via Oculus Rift. They were represented as a giant head floating over the map. But the panda and I were invisible to them, tiny figures on a stage acting as bystanders to their gameplay. With a few little cameramen, we could potentially be sharing analysis with a live audience and providing a kind of virtual sideline commentary.

Hooper says VReal runs locally, meaning that each viewer’s computer is responsible for re-rendering the game and presenting them with their unique, virtual experience. By lightening the hardware burden on the streamer, VReal’s YouTube-like service is theoretically able to admit an unlimited number of viewers to any live VR broadcast.

Imagine an audience of thousands of people, virtually inside the same VR experience alongside you. Streamers will be able to keep their chat window visible at all times, and can reach out and allow viewers into the live game world with them. Instead of pushing a button to sponsor your favorite Twitch personality, with VReal you could theoretically hand them a digital wad of cash.

In addition to a receipt, you could receive a virtual high-five in return.

Looking down on a tiny spectator standing on an in-game VReal stage.

"A lot of people ask ... what are the technical constraints on the design, and do we make it scale?" Hooper says. "If you look at traditional multiplayer technology, it doesn’t scale much beyond 16 or 32 players unless you go to the MMO route. So really, VReal should scale like Twitch or YouTube with thousands of people watching a stream.

"But what we don’t attempt to do is, we don’t attempt to render the entire audience for you. From your perspective, you’re in a stream there and with a famous streamer that you’re interested in watching. You’ll see your friends who are watching with you in the stream. You’ll see the avatars for your friends or who you choose to have present. But we’re not going to try and render all 1,000 people in the stream. Now from that streamer’s perspective they’ll see a number that lets them know that thousands of people are watching, but they’ll just see the ones that they want to."

If the technology works at all like Hooper has described it, it will be very exciting to see it in the wild. He says that his team will have more later in the year when the beta of the service goes live.