How video games and augmented reality could cure phantom pain

For the most part, phantom limbs and the pain they can feel remain a medical mystery. No one is quite sure why a person who has lost a limb might feel like they can move it or be tormented by stabbing muscle cramps and searing pain in that nonexistent appendage.

But a researcher in Sweden is getting closer to finding a cure for phantom pain thanks to an experimental muscle mapping treatment that uses augmented reality to trick the brain into thinking a missing arm is back, and video games that have a player steer a car with a missing hand.

Max Ortiz-Catalan is a PhD candidate at Chalmers University of Technology and the Centre of Orthopaedic Osseointegration at Sahlgrenska University Hospital in Sweden with a master's in science. He designed a system that combines advanced myoelectric pattern recognition with augmented reality and video games.

Essentially, Ortiz-Catalan attaches probes to the point where an amputee's arm was removed to detect muscle movements. That data is then fed into open-source computer software Ortiz-Catalan created, which deciphers those movements and uses them to create a computer-generated image of the missing arm. The patient then uses the muscle movements he would usually use to move his missing limb, and the augmented video shows the reattached arm moving in real time.

In the next phase of the weekly two-hour treatment, Ortiz-Catalan uses the same electrodes to have the patient steer a car in video game TrackMania with his missing hand.

Both treatments seem to have a massive impact on the patient, who had spent decades dealing with a constant burning pain in his missing arm. Within weeks the pain lessoned and the phantom limb, which felt to the patient like a clenched fist, gradually relaxed to match the computer image he saw on the screen of an arm with an open hand. While the system works, it's still unclear why, though Ortiz-Catalan has a theory.

"When you have a part of your body amputated, it doesn't matter if it's a limb, a nose or a breast, some people have sensations of still having that part of the body, phantom sensations," he told Polygon. "When people perceive pain in that part of the body it's called phantom limb pain. It feels like burning or stabbing, or maybe like the wrist is extremely bent back. Nobody knows the underlying mechanism of phantom pain.

"One theory is that it has to do with the cortical map, which has a representation of every part of your body for motor control and sensation. What researchers have found when people are missing a part of the body, that part of the body stops being used for motor control and there is a reorganization in that cortical area, and something happens in that reorganization that causes pain. We think the pain is related to the cortical reorganization."

The original treatment based on this theory was called mirror therapy. It made use of a mirror box, allowing a patient to see the missing limb in a mirror and thereby trick the brain into thinking the limb was back, and reducing the pain.

"The visual feedback and the motor execution helped to restore the cortical map, which gets rid of the pain," Ortiz-Catalan said.

The problem with mirror therapy is that it doesn't work for bilateral amputees, like people who might be missing both legs. It also could be easier to set up than designing a mirror big enough to work for certain cases.

"Some say, 'I can move my little finger' or ‘I can move my wrist.'"

"In ours you don't need to have your other limb because the movement comes from signals from the stump," Ortiz-Catalan said. "You need to work out the muscles in the stump to make it work. With augmented reality, you see yourself and we put the limb where the limb is missing."

The other reason Ortiz-Catalan uses the augmented reality first is to help map the impulses in a patient's limb.

"The first thing we check is what they feel they have control over in their missing limb," he said. "Some say, 'I can move my little finger,' or 'I can move my wrist.' Every patient has a different sensation.

"I ask them to do different motions and then see how good our algorithms are at predicting signals."

Once they feel like the signals are correctly mapped, the patient is asked to play racing game TrackMania.

Ortiz-Catalan decided to add video game playing to his therapy because it is a fun way to get people to a lot of specific motion execution, he said.

"We ask them to play games using the missing limb," he said. "At the end of the day, making them use that brain area, which they are not used to using, potentially helps to restore the brain map."

While Ortiz-Catalan said he could likely design a system that would allow players to play other games, even something as complex as a first-person shooter like Call of Duty, he doesn't think it would be as useful as having them use racing games.

"We think the racing games are quite good because you can imagine rotating your wrist and the car is going left and right," he said.

The system has a patient turning the phantom hand left or right to control the steering of the car, flexing the elbow to accelerate and extending the elbow to brake.

Now that Ortiz-Catalan has completed an initial case study on a single patient, his plan is to extend the treatment to clinical trials at four hospitals, three in Sweden and one in an undisclosed location, he said. His plan is for those trials to last for a couple of months and include 15 to 20 patients, with the goal of having the results by the end of the year.

His long-term hope is to get the technology into hospitals around the world, and perhaps, into people's homes so they can do treatment on their own. He also sees the technique being used for treatment of other conditions, including strokes and spinal cord injuries, and says he's very open to collaborating with other international researchers.

"It's a rehabilitation tool for neuromuscular rehabilitation," he said. "It makes movement fun to do. If you have an injury and they tell you to move the shoulder 20 times, you may not do that. But what if instead of that you could hook up this system, and every time you do your rehabilitation you're driving a car?"

While this isn't the first time video games have been used for rehabilitation, he thinks it's much more effective than using something like motion controllers driven by Nintendo's Wii, the PlayStation's camera or Xbox's Kinect.

"A lot of people have started using Kinect and other gaming consoles for rehabilitation, and they have lost the focus of rehabilitation and are more about playing games," he said. "It now is very common that patients try to recover with video games, but they're more focused on achieving the goals of the game and not the rehab."

Also, Ortiz-Catalan points out, motion control games can't predict the movements of an amputee's phantom limb.

"That technology would not allow you to know when a patient wants to move their arm," he said. "We can't take a hammer and solve all of the problems with the hammer because that's what we happen to have. Sometimes we have to see what the problem is and create a new tool to solve the problem."

Good Game is an internationally syndicated weekly news and opinion column about the big stories of the week in the gaming industry and its bigger impact on things to come. Brian Crecente is a founding News Editor of Polygon.