Kinect used in sign-language recognition and translation

In a collaboration with Microsoft Research Asia and the Institute of Computing Technology at the Chinese Academy of Sciences (CAS) researchers tested how the Kinect's body-tracking features can be used in sign-language recognition, according to the official Inside Microsoft Research website.

According to the blog, initial results show that the technology built from Kinect for Windows can help those who use sign language as primary language to better interact with computers, similar to speech recognition software.

The system features a Translation Mode that translates sign language into text or speech, which currently supports American sign language with the capacity to support other sign language varieties.

Using an avatar, the system's Communications Mode allows a deaf or hard-of-hearing person to communicate with a hearing person. The avatar acts out the text input from a keyboard and when the deaf or hard-of-hearing person responds with sign language, it is converted into text.

"From our point of view," says CAS Professor Xilin Chen told the blog. "The most significant contribution is that the project demonstrates the possibility of sign-language recognition with readily available, low-cost 3-D and 2-D sensors."

The research features in the paper "Sign Language Recognition and Translation with Kinect." The paper was co-authored by CAS researchers Xiujuan Chai, Guang Li, Yushun Lin, Zhihao Xu, Yili Tang, and Chen, as well as principal researcher at Microsoft Research Asia, Ming Zhou. The project was supported by Microsoft Research Connections.

You can watch the technology in action in the video above.

More from Polygon

Hearthstone - Plague Quarter gameplay video

  • A guided video tour of an Unreal Tournament map

  • Freedom Planet gameplay video

  • Adventure Time trailer

  • Hearthstone: Naxxramas Overview

Latest Discussions

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.
Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_5353_tracker