On eve of VR launches, hands-on with a new haptic ideal

I arrived at the University of Pennsylvania's Levine Hall on the same day as a major AR / VR vendor, the week before the 2016 Game Developers Conference. I wasn't told what vendor this was — though I'm told the company has NDAs signed with eight separate vendors and, if you count up the companies active in the space, there aren't too many. We were there to see a product from a company called Tactai that promised to solve one of the space's most pressing issues: the absence of touch.

Why UPenn? Because Tactai's technology is based on the work of Dr. Katherine Kuchenbecker, a tenured associate professor of mechanical engineering and applied mechanics at the Ivy League university, whose research on haptics and touch made her an expert in the space years before virtual reality would give it new purpose. See, for example, this profile from Popular Science that listed Kuchenbecker in its "Brilliant 10," or this 2012 TED presentation titled "The Technology of Touch."

"This work is based on research that goes all the way back to my PhD, where we set out to make the most realistic virtual objects anyone had ever made," Kuchenbecker told me. "And in my early years here as a professor, I had a grant from the National Science Foundation on what I call haptography — haptic photography — to capture how the surfaces feel to the touch and then recreate them."

After a brief introduction to the company by CEO Steven Domenikos, Kuchenbecker walked me through two haptographic demos before we arrived at the product that I — and those other eight companies that preceded me that day — were there to see: a small, humble 3D-printed device that attached to the index finger, not unlike a fingertip pulse monitor. It attached to my finger with velcro, and was adjusted so my fingertip hovered just above a single actuator. It was connected to some lab equipment by wire and was tracked in 3D space by a magnetic field.

"It’s one motor that both lifts the platform in and out of contact with your finger and presses to varying degrees," Kuchenbecker explained.

I then put on an Oculus Rift prototype headset.

The demo was a single room, with five objects inside. From left to right: a lamp, a nut (think bolt, not acorn), a wooden box, a bag of flour and a moai, one of the iconic Easter Island statues. My finger hovering in the virtual room, I started with the statue.

Did it feel exactly like touching a rock statue? Well ... no, not exactly. But it did in the same way that when you're in virtual reality you can look around and see things, although you understand it's not the same thing as being on a spaceship. Or on an alien planet. But it is convincing if you allow it to be. While I rubbed my finger across the stone statue, and felt the actuator tap against my fingertip in a pattern that "matched" that of touching actual rock, my eyes saw a virtual finger stop against a virtual rock. Pushing further increased the force of the feedback, but my finger didn't penetrate the statue. While there was nothing in the actual room preventing my arm from moving forward, the effect of having my finger "stopped" was definitely convincing.

The bag of flour had a smooth, papery texture and was a little squishy. The wooden box was solid and a little scratchy. And then the nut ... while my virtual finger stopped when I "touched" it, it had no haptic feedback. It was immediately disappointing. I could tap it, but I couldn't feel it. And finally, when I touched the lamp, my hand passed through it entirely.

The demo, like the technology itself, was fairly simple. There was nothing else to it. There was no binaural audio allowing me to hear the empty box versus the solid rock. There was no other movement in the space. Even the texture and graphical quality of the demo were humble. I imagine this haptic experience coupled with a higher production value would only further cement the illusion.

The simplicity of the device was the most impressive element for me. The second demo Kuchenbecker showed me before we got to the VR touch demo used something very similar to a Novint Falcon (pictured below). If you've never heard of this particular device, that's because it never achieved any semblance of mainstream adoption. I first saw it at E3 ... in 2008, and a steep price and cumbersome hardware combined to ensure limited (if any!) game support. No matter how impressive a piece of technology is, it's dead in the water without developer support. And no matter how much developers may want, say, touch controls in their games, they won't waste the time without consumer support.

Tactai imagines a world where its technology makes it into a hardware partner's device and, using its SDK, the textures in the game are matched to existing haptic profiles in its library.

"Currently we have 100 textures scanned. But there's really no limit," Kuchenbecker said.

Tactai is now raising funds to take this humble prototype to completion. "We're fundraising for $30 million series A," Domenikos said. "We have quite a bit of interest on that from a combination of investors as well as industrial partners."

Whether Tactai's technology finds its way into a consumer product has yet to be determined. Whether it finds its way into a product that you or I would ever use is another question; Tactai isn't shy about its non-gaming ambitions — "clearly, gaming is an easy answer early on," Domenikos said — but neither is Oculus. Facebook imagines VR's future in much grander terms than simply video game technology; that's just where it starts.

It's early days for VR, with the retail release of Oculus Rift just a few days away, the HTC Vive just behind it and PlayStation VR later this year. And even these are first-generation products, years away from critical mass. I can imagine the kind of technological feature brinksmanship that will help propel one, or all, of these solutions into more households. And if the price is low enough and the support simple enough, I can absolutely see touch, and perhaps Tactai's solution for touch, be a part of that.

Update: Clarified that in fact only one AR / VR vendor had visited the same day as me, though Domenikos noted "more will be visiting us over the next a couple of weeks." Also noted that while the lab equipment looked and functioned like a Novint Falcon — see above picture — it was not, in fact, that exact product.