Microsoft's hologram glasses change the way you see the world

Microsoft's hologram headset is real, is absolutely coming to market, according to the company, and, for the most part, does what it says it does: creates interactive holograms in a user's view of the real world.

Early builds of the HoloLens device allowed me to walk on the surface of Mars and talk to a NASA scientist about the formation of the rocks there. A Microsoft employee used Skype and the headset to show me, with diagrams layered over my view of the wall in front of me, how to wire a light switch. I watched someone build a 3D model of a toy in real time and, finally, had a chance to play a take on Minecraft that involved digging holes through coffee tables and blowing away a section of a wall to reveal the lava-scape behind it.

There is no price and the window for the release date is so wide it's almost nonsense (sometime during the lifetime of Windows 10 — though they won't say if that means before the launch of the next operating system). But Microsoft's Phil Spencer assures me that this is a device Microsoft is serious about releasing.

"We wouldn't be showing it," he said. "It wouldn't have a name, if we weren't going to release it."

Microsoft seems exceptionally proud of how well it was able to keep the details of HoloLens a secret. The company even bragged about it during the day's keynote. The lab used to build the unit and then to start creating applications for it is actually located in a subterranean office complex beneath the cafeteria of the Microsoft campus welcome center.

Microsoft

Today is the first day a large group of outsiders have been allowed in to see the kits and play with the technology, and the team seemed nervous.

Spencer said the group responsible for HoloLens and its software watched the press conference unveiling as a group in a different building earlier today. The group then made their way over to the demo stations.

Corporate vice president of Microsoft Studios Kudo Tsunoda, who suddenly disappeared from public appearances at Microsoft events years ago, tapped me on the shoulder right after the keynote ended.

He wanted to know what I thought. He left quickly after saying Hi.

"He's nervous, we're all nervous," Spencer said later.

As the time rolled up for my group's chance to see HoloLens in action, we were told that we couldn't bring any bags with us to the demo rooms. Or video or photography equipment. Or audio recorders. We were allowed to bring a pad of paper and pen or pencil.

We were brought down in groups of a half dozen or so and then walked through a quick primer on the tech and how delicate it is in its current, nascent form.

The technology, we're told, uses gaze, gesture and voice interaction to work.

Your head movement and eye movement is essentially how you control the cursor, as if you were using your head to move a mouse. And then to click, you raise a hand in front of you and move your pointer finger down as if flicking off a light switch.

Dropping In

The HoloLens development kit looks a lot like something you might expect to see in a Terry Gilliam movie about a future that fetishizes technology, something that looks like it was built by a tinkerer in their garage with scraps.

The augmented reality hologram display unit is, in its current workable form, two pieces of hardware. One is a block of batteries, computer parts and fans which hangs from your neck by a strap, providing the headset with both the electrical and computational power it needs to work.

The other is the headset, a device built of wired and metal frame so delicate the user is not allowed to touch it, instead a helper places it on your head.

Before using the device you need to have your interpupulary distance measured and then gently maneuver the two lens on your nose until the display shows correctly in front of your eyes.

Examined closely, the dev kit appears to be a pair of abnormally thick glasses attached to a wobbly headset with wires and computer parts stuck to it.

It doesn't look real, it looks movie real, like maybe something Doc would create to help you travel through time or something you'd be forced to wear at your job in a dystopian future.

The effect of wearing the glasses also doesn't feel real. But not because it seems fake, rather because it is initially a bit unbelievable.

When the headset is first placed on your face, you see a semi-transparent rectangle, about the size of a playing card, floating several inches in front of your view. In the center of the rectangle is the new Windows logo. Your view around the rectangle is completely unobstructed, so it feels a bit like an oversized heads-up display.

Onsight

My time with the HoloLens was spaced out across four demos.

The first experience took place in an office with a cleared out space in the center. After adjusting the headset and standing, the team dropped me into a view of Mars.

Unlike virtual reality, it didn't feel like I was submerged in an entirely created world. Instead if felt like I had a detailed, expansive 3D view of Mars in front of me and as I walk around and turn, my view reacts accordingly. The view was so detailed that it all but blocked out my view of anything behind it, though I could still glimpse my real surroundings in the periphery of my view.

This particular view of Mars was one pulled from images captured by NASA, it wasn't fictional, or even generated, it was me looking at the actual landscape of Mars from the surface of Mars.

It looked a bit like an Arizona desert, but with a lander standing nearby.

After looking around and walking around for a few seconds, the team had me walk over to a real nearby computer sitting on a real desk in the real office. My view of Mars was still there, but the device knew not to obstruct the computer, so it looked as if the computer and the desk were sitting on the surface of Mars with me.

The team showed me the flattened images scientists usually use to plan expeditions and run tests on the surface of Mars. Then they brought up another program that allowed me to drop flags on the computer screen's map and have those flags show up in my 3D view of the planet.

Then one of the team told me to move the mouse over, all the way over, to the extreme edge of the screen. When I did the mouse pointer jumped from the edge of my computer screen onto the surface of Mars. It was surreal and a little startling. I was able to move the mouse across the planet's surface, dropping flags in real time, in real locations.

Next we made a call to Jeff Norris, a real NASA scientist who was remotely helping Microsoft show off the tech. He dropped into my view of the world as a vaguely shaped avatar. A dotted yellow line showed in real-time where he was looking. We chatted for a bit and he pointed out things in the rocks using his gaze and then asked me to issue commands to the rover using my gaze.

It was simple, effective, profound.

Skype

I know how to rewire a light switch. It's actually one of the few things I know how to do around the house. But the team in my second demo asked me to humor them.

"Pretend you don't and let us show you how to."

Headset on, I used my finger to place a Skype call to a remote Microsoft employee. He answered and asked me to have a seat next to the wall.

Then he asked me to look up at a rectangular hole in the wall where three exposed wires jutted out.

"I'm seeing everything you see," he told me. "Now look to the right of the hole. Yeah, right there. Pin me."

I used a finger click to select a pushpin symbol in Skype, pinning the square that contained the call and my helper's live video feed to the wall.

"Now look down at the tools on the desk."

My helper drew arrows in the air above the tools, explaining what each tool was used for. Next he drew a diagram next to the hole in the wall to explain how I was going to install the light switch.

The whole process was slick, seemingly natural, offering an easy way for someone to walk a friend through what could be a complex set of directions as if they were there.

While the HoloLens was using a special build of Skype, the folks in the demo room said that the Windows 10 version of Skype would support calls to HoloLens once it's out, including the ability to draw on live images and look through a person's eyes.

Holo Studio

The least interesting of the HoloLens demos was a look at the HoloStudio, an application designed to allow anyone to quickly slap shapes together to create 3D images.

During the short demo, a producer on the project quickly created a fairly well-crafted 3D Koala, a monster truck, and then turned the Koala into a space Koala.

The big draw for the studio is that people will be able to easily send off their custom designs to a service that could 3D print the object and ship it to your house.

It was a demo designed to show you how HoloLens can help you build 3D in 3D. It was also the only demo that press weren't allowed to interact with.

Holobuilder

The final demo was the most relevant to gamers: Holobuilder.

While the demo looks a lot like Minecraft, the team was quick to point out it was inspired by Minecraft.

After placing on the headset, I was asked to walk to a back corner of a room made up to look like a den. With a finger click, the device starting scanning the room, waves of rocks pulsing out away from me as it took in the objects in the room. Then it dropped Minecraft buildings, castles, caves, walls into the room I was standing in.

The coffee table was home to a small farm, a castle and a waterfall that fell through the center of the table. Standing over it, I could look down into the cave below. Looking under the table I could see parts of the cave and the hole.

A couch had a small house on it, creepers walking around, while another table was home to another homestead.

The helper in the room asked me to walk over to the wall where there was a house on one side of a low table and a group of creepers on the other side. By saying "shovel" I was able to turn my tap gesture into the action of digging into a block. I quickly dug a channel in front of the house to protect it. I noticed that the cave beneath the house was starting to light up as sun streamed in from the holes I was digging in the ground above.

Later, I set off a series of in-game explosives attached to the real wall of the room. The explosion blew a hole in my view of the wall, exposing an entire Minecraft-like chamber of lava, red stone and darkness beyond.

You can, I was told, dig away or blow up every wall, the ceiling and floor in the game and find the world of this Minecraft-inspired title beneath it.

Later, Spencer told me that while the game was only inspired by Minecraft, Jens "Jeb" Bergensten, the lead developer at Mojang on Minecraft, had a chance to see the game in action during a visit to the company's campus.

"He loved it," Spencer said. "It really energized him."

Holograms?

"Are you ready to go play with some holograms?"

"We're going to experience holograms first hand!"

But we didn't, not really.

Microsoft is being a bit coy about the technology behind their HoloLens headset, but it's no hologram, a thing made through the manipulation of laser and lights.

Augmented reality is probably a better description, though I heard Microsoft folks referring to it as a mixed reality experience. And a lot of what the final experience will look like has yet to be nailed down.

That image size, the one that only seems to fill the center of your view, isn't yet finalized, I'm told, nor is the price, or a hard release date or the software that will become a real thing.

While OnSight is real and will soon be in NASA's hands, none of the other stuff is.

What is set in stone is that the unit will be a single unit, a device all built into one headset with no wires, no need to connect to a computer. HoloLens will be a stand-alone device that uses the computer built into it to deliver its experiences, Microsoft officials said.

It's also very much different from the sort of experience you have using a virtual reality headset like the Oculus Rift.

HoloLens is only a competitor with Oculus in the sense that the general public may only have the attention span and money for one crazy, over-the-top headset that changes the way your view your world.

I still don't know which camp I'd fall in if I had to choose today: virtual reality or mixed reality, but the HoloLens does seems more functional, more useful across a broader range of experiences right now.

That and it let me stand on the surface of Mars and talk to a NASA scientist.