How Sony is experimenting with the DualShock 4 for Project Morpheus VR

Bringing a virtual DualShock 4 controller into the VR environment of Sony's Project Morpheus can help immerse players in the experience, as well as give them a helpful reference point, Sony Japan Studio's Nicolas Doucet said at GDC today.

During a technical presentation on Project Morpheus, Doucet showed off how the studio is experimenting with the PlayStation 4 controller in a VR setting, including augmenting the controller in VR to make it behave beyond how it could in the real world.

Doucet, a producer who works in Sony Japan Studio's internal development department, worked on The Playroom for PlayStation 4, a suite of demos and minigames that showcase the capabilities of the PlayStation Camera and DualShock 4. During his talk at GDC, he showed how the DualShock 4, PlayStation Camera and Morpheus headset can work in concert to turn the PS4 controller into a flying ship, a robot-launching device, a set of handlebars for a motorcycle or even a virtual MiniDisc player (Sony in-jokes were rife throughout the demos).

Sony Japan Studio has developed a series of graphically simple demos to explore how the DualShock 4 works in a VR setting. Some focus on using the controller's touchpad. In one demo, players are placed in a darkened room and can pull virtual light switches with a downward swipe of the touchpad, or open sliding doors by swiping left and right. An upward flicking motion, Doucet said, is good for throwing things — basketballs, pizzas, shuriken. A pulling apart motion is an intuitive input for opening doors.

Using the DualShock 4's light bar and motion sensing, players find it easy to look around and explore their environment, even if they can't see their hands, Doucet said.

Doucet added that players don't need to interact with a simple one-to-one recreation of the DualShock 4 in virtual reality. In one demo, the virtual controller sprouted jet engines from its handles and a player flew a winged version of the gamepad through rings and between obstacles, like it was a game of StarFox. In another, the virtual DualShock 4 touchpad was like a door that opened up, revealing a Playroom robot who popped up and stood on the controller. It kept eye contact with the player, making it feel more alive.

Another demo used the virtual DualShock 4 controller like a proton pack from Ghostbusters. Players could track and ensnare a ghost with a beam that projected out of the light bar. The ghost was then sucked into the virtual controller and could be seen trapped behind the touchscreen panel.

Players quickly adapt to DualShock 4 controls in VR, Doucet said. A method that helps ground players is "snapping" a controller to an in-game object. The demo showed the controller being locked into a crane game cabinet and later to a motorcycle, letting players steer in VR by tilting left and right. Players can then detach the virtual controller from an in-game object by yanking it out.

Using some of these natural movements with the VR representation of the DualShock 4 can help sell the sensation of being in the virtual environment. It can make players feel like they're really flying, Doucet said, but comes with some concerns, namely motion sickness. He said developers should let players take it slow and give them wide turns in VR experiences like flying. Sony Japan Studio is still experimenting with how to convey collisions in VR, because they can be jarring.

Doucet ended his series of demos with a look at something more polished. He used a virtual DualShock 4 in Project Morpheus to interact with a bunch of Playroom robots who burst forth from the controller's touchpad. He could play music for them by pulling a virtual MiniDisc cartridge out of the controller and sliding it into the light bar area. That light bar could also be used as an intense spotlight that he shined in the faces of those bots, who squinted in response. Like earlier Playroom demos, it was very, very cute.

We also got a look at a VR dollhouse that was populated with Playroom robots. This virtual diorama featured bots that were going about their business — watching TV, washing cars, playing Frisbee — but started to act differently when the player's focus was on them. It was an interesting display of subtle interaction, one that the player might not even know they were "controlling," Doucet said.