Indian Students Develop Device That Converts Sign Language Into Speech And Text – Lending Voice To The Mute

A team of Indian students has successfully developed a device that dynamically translates sign-language gestures into speech and text.

The students of the Holy Grace Academy of Engineering, Mala, have developed a device which is able to understand the gestures involved in communicating with sign-language into simple words that are “spoken” by the device. This can significantly alter the way we communicate with the speech impaired.

The four Electronics and Communication students have named the revolutionary device “Gesture Vocalizer,” which aims at shortening the time required to understand what is being communicated. The team wanted to minimize the communication gap between the speech impaired and common people, said Minu Varghese, who developed the device along with S. Deepthi, Delna Domini, and Nimya Varghese.

“The device is based on body positioning technique (mainly hand gestures). The aim of this system is to make a simple prototype by taking the gestures and converting it into audio-visual format so that it can be understood by everyone.”

Essentially, the device consists of three components that work in tandem to understand what the speech-impaired person is trying to communicate, then translate the same into spoken language, and finally convey the same via robotic speech or text. From a technical perspective, this involves an input component which consists of a glove that’s covered with sensors that decipher the location and movement of the hand. The team explains the glove has sensors to determine the bending of fingers, and an accelerometer that measures the tilting of the hand.

The data obtained from these sensors is then sent to the component that compares the obtained values to the ones in the database to “understand” what the person is saying. Sign language is universal, and the gestures are common everywhere. Hence, the all-girls team did not face any trouble sourcing the data to optimize the algorithm that translates the gestures into spoken language. The translated speech is used to produce the corresponding pre-recorded output messages in the form of audio as well as text that’s displayed on a screen.

Considering the fact that their Gesture Vocalizer needs to be portable so that those with speech-impairment are able to take it anywhere and “talk” to anyone, the entire device works on a 9V radio battery.

Sign language may not be hard to learn, but this technology surely makes communication with a speech-impaired person really simple and efficient, thereby ensuring they are able to communicate their ideas faster.

[Update] Way back in 2012, Ukrainian students had developed a similar device, which they claimed could be manufactured for just $75. But the device hasn’t made it to market.

[Image Credit: Holy Grace Academy of Engineering, Mala, via The Hindu]