Researchers at Cornell University have developed a new AI-powered ring that can track American Sign Language (ASL) fingerspelling in real time. The device, called SpellRing, uses micro-sonar technology housed in a 3D-printed ring no larger than a quarter to capture hand and finger movements.
The ring contains a microphone, speaker, and mini gyroscope to track hand motions and process them through a deep-learning algorithm. Testing with 20 ASL users demonstrated accuracy rates between 82% and 92% when tracking more than 20,000 words of varying lengths.
“Many other technologies that recognize fingerspelling in ASL have not been adopted by the deaf and hard-of-hearing community because the hardware is bulky and impractical,” said Hyunchul Lim, a doctoral student in information science at Cornell and the study’s lead author. The research will be presented at the Association of Computing Machinery’s conference on Human Factors in Computing Systems in Yokohama, Japan, from April 26 to May 1.
The development team faced significant challenges in training the AI system to recognize the 26 handshapes corresponding to letters, as signers often modify letter forms for efficiency and flow. The current version of SpellRing can be used to enter text into computers or smartphones via fingerspelling, which is primarily used for proper nouns, names, and technical terms in ASL.
The research team acknowledges that fingerspelling represents only a portion of ASL communication. “Fingerspelling, while nuanced and challenging to track from a technical perspective, comprises but a fraction of ASL and is not representative of ASL as a language,” noted Jane Lu, a doctoral student in linguistics and co-author of the study. Future development plans include integrating the micro-sonar system into eyeglasses to capture upper body movements and facial expressions.
The project, funded by the National Science Foundation, was developed by researchers in Cornell’s Smart Computer Interfaces for Future Interactions Lab within the Ann S. Bowers College of Computing and Information Science. The team worked closely with experienced and novice ASL signers throughout the development process to ensure practical functionality.
Source: news.cornell.edu