How Fantastic Contraption has made virtual reality social

Virtual reality, despite what is often written, is a social experience. But you have to spend some time with the technology in order to understand how strapping a headset across your eyes can bring you closer to other people in the room.

"There was a lot of press about a year ago going around about VR being an isolating experience, but every time we put someone into the game and we have the TV in the living room that just showed the output from the headset we had people on the couch watching that stream and cheering and hooting and hollering and they loved it," Andy Moore, one of the developers working on Fantastic Contraption, told Polygon. "They were part of the experience. It’s like Pictionary, you have one person up there acting but everyone else is a part of the experience."

Fantastic Contraption is a virtual reality version of the existing 2D game, and it allows players to create goofy, virtual machines and structures with the Vive's motion controls. It's one of the more exciting games coming to the platform. It's also one of the most enjoyable games to play with others.

That social aspect of the game wasn't anything they had anticipated, but they quickly saw that it was an opportunity to do something special. "It caught us by surprise, but it caught us by surprise in the first week," Moore said. They got to work creating something unique to help bring players together.

This is code that's being shared

One of the things that was striking during my time at Valve's content showcase for the Vive was how many games are including a third-person camera that allows other people in the room not just to see through the eyes of the player, but to see a view of the entire environment. That ability to watch the player as if they were a character in the game is an important part of making VR social, and many teams brought up Fantastic Contraption as an inspiration.

"We look at the Northways, who are doing Fantastic Contraption, and that mixed-reality streaming they’re doing on Twitch is such an important piece in helping to communicate what room-scale VR is," Patrick Hackett, a VR developer of Google’s VR team told Polygon. They've since added a third-person camera to Tilt Brush so you can watch the player paint in real time.

If you haven't seen the Twitch streams Northway Games has created for Fantastic Contraption, allow us a moment to blow your mind. The team has created a way to livestream the game from within virtual reality itself so you can see the player, the environment and also other individuals in the room. It gives an immediately understandable impression of how the game looks and plays. The play begins at around the 20-minute mark.

I caught up with the Northways over a video call to find out how they achieved this effect.

"You have your webcam, and a camera that only shows you foreground objects, so anything that’s in front of the head-mounted display, and that’s a game camera, and you have a game camera that only shows you background objects that are behind the HMD," Sarah Northway explained. "The foreground one fills in the blanks with a green or gray color that we then make transparent, and the webcam has a green screen that we then make transparent, so you have transparent layer, transparent layer, background, and we sandwich the three together and that’s how you get the world."

So that's two in-game virtual cameras, one real-world camera, and then they create a single video stream of all three sources in the popular streaming software OBS by laying them all on top of each other and then streaming that video signal. "We just got a 4K monitor so we can do 1080p with all these windows at once," Northway said.

They're working on making the process simpler for others, but for now the system does require a rather large room and a good amount of green fabric. It's not tricky technology for people to set up, but there's a bit of math to it, and it can be a bit "fidgety," as one developer described it.

You can take a look at the image below and manipulate the slider to see "real" life, and the mixed reality stream.

The mixed reality streaming is impressive, and the process to create it to stream inside of virtual reality will only get simpler with time. For now the code they've added to the game, and are sharing with other developers, allows anyone to set up a third-person camera so others can get a sense for the entire scene while they're watching someone else play a game in virtual reality, allowing the entire room to participate.

This work, and the shared code, has already proven influential in the virtual reality community as other teams try to make sure their games can be played socially.

The technology on display here has already gone a long way to explaining virtual reality to people who don't have an easy way of finding a demo to try themselves, and it will hopefully help to get rid of the idea that virtual reality is something that is played alone, and isolates the player.

During the content showcase it was a blast to watch others play games, and to have them watch you. People react to each other's moves and demos, and people were sharing stories. Virtual reality is, although it can seem hard to believe until you've spend time in the hardware, a technology that is proving surprisingly effective at bringing people together.

Colin Northway has some thoughts on why that happens.

"When you watch a movie you know it’s not real and the characters in it are not real, it’s the same kind of thing watching someone in VR," Colin Northway said. "The person in the headset is allowing themselves to do one kind of suspension of disbelief, they know they’re in a living room and they think they’re someplace else in a fantastical world. As you watch a screen with them playing you empathize with where they are, and you believe it and you can see through their eyes and you know what’s going on."

It's not just a matter of knowing what they're experiencing, you can put yourself in their shoes. "You share that experience with them, you know what world they’re in. When they see something crazy so do you, and you understand how they’re reacting to that," Northway continued. "When an entire room gets to see through your eyes, you have this great communal experience."

And the work the Northways are doing is finding a home in multiple games. I asked if they were ever hesitant about sharing their ideas or code with other developers, and they both waved away any concerns.

"Indie has always been this way, all indies are open about sharing and doing stuff. We never really see ourselves as competitors," Colin Northway said. "If people get really excited about Job Simulator, and that makes them more likely to invest in and spend money and time to set up room-scale VR then we’ll be there when they want to play something else. A win for any of the games is a win for all of us."

And with things like these streams and the third-person cameras that are shipping with many of the games coming to the Vive, VR has become a bit friendlier.