The real-life holodeck didn't work quite as well as advertised

Dhan Balachand is the CEO of Sulon Technologies, and I’ve been lured into a hotel room at GDC to check out something that’s being billed as a real-life version of the holodeck from Star Trek.

The product is called the Cortex, and it delivers a combination of augmented and virtual reality.

"They’re not mutually exclusive," Balachand said. "They can combine to create some really crazy experiences."

The 30 minute demo was plagued with technical difficulties, but as a proof of concept the Cortex was able to deliver what was promised, although the execution has a ways to go before it becomes an attractive product.

Some assembly required

The head mounted display doesn’t actually come with a display, you have to provide your own by sliding in an Android device with a six inch or larger screen. The hardware syncs with the rest of the systems in the helmet to deliver an experience that’s frankly very, very strange.

There is one motorized camera on top of the helmet that senses the size of the room, you can feel it move as it looks around, and a front-facing camera provides a real-time video feed of your environment. You can play a game in augmented reality mode, where the camera shows you the real-world room and super-imposing floating targets or monsters for you to shoot, or you can enter a fully virtual reality room to explore.

Here’s the kicker: Since the hardware’s camera knows the size of the actual room, you get to walk around the environment itself and explore. The virtual room scales up and down to match your surroundings. You reach out to touch the virtual wall and feel the real wall, and a proximity sensor helpfully dings when you move too close to a real-world object.

"You build your game scene as you wish, and the VR engine takes that game that you just built and it applies geometric customization so that it scales," Balanchand explained. The system was plugged into a power supply so the wires were an issue in the prototype, but the renders of the development kit show a much smaller and completely self-contained product.

I wasn’t able to demo the feature, but I was also told that the helmet was smart enough to change transparency in real-time if someone enters the room. So if you’re running around a virtual dungeon that’s exactly the size of your bedroom and one of your kid walks in, the display may shift to show some of your real-world environment so you don’t trip over a toddler. Getting used to walking around the augmented reality environment takes a bit, as the camera seems to show a slightly zoomed-in version of the real-world.

Latency was a problem; the system had a hard time keeping up with even moderately fast head movements. They promised that the spatial scanner they created for the dev kit is ten times faster than the unit I was using. The system also broke down a number of times due to a loose cable, but that sort of mishap isn’t rare in early prototypes of this sort of display.

The helmet I was using was a very early prototype, they assured me, and the development kit will be much easier to use while still being smaller and offering a smoother virtual reality experience. While the use of the spatial sensors to "paint" fantastical environments over the actual room was impressive, as a virtual reality experience the screen wasn’t up to the task. The video embedded in this story is largely aspirational.

The $499.99 development kit is expected to ship out in the fourth quarter of this year, and you can see the vast improvements they're hoping to see in the hardware by comparing the prototype image with the render of the development kit. The company is also expected to ship with their own solution for three-dimensional controllers, although the prototype used the Razer Hydra.

The idea of a wearable holodeck is interesting, and certain aspects of the technology were very impressive. Still, there’s going to need to be vast improvements in the hardware and software before this becomes a technology to watch.