An Approach to Real-Time Indian Sign Language Recognition and Braille Script Translation

There is a need to bridge the communication gap between the speech or vision impaired and the rest of society. The limitation associated with existing solutions is that they don't perform well in the real-time scenario. Also, they yield a low level of accuracy. To mitigate these limitations, we aim to develop a system that recognizes the hand gestures for the English alphabet in Indian Sign Language (ISL) and then translates them into Braille Script in real time. The proposed mechanism can identify 26 hand poses for the 26 letters of the English alphabet as per the ISL. The different signs are captured from a camera and its frames are then used for recognition using Convolutional Neural Networks (CNN). In our work, we also utilize lookup maps for Braille Script Translation. The proposed method affirms the efficacy of our self-acquired dataset with an accuracy of 94.23% on the Sign-Language-Custom (SLC) model. Next, an accuracy score of 1 is obtained on the C-O and M-N models designed to eliminate the ambiguity in the SLC model for the above-mentioned two pairs of alphabets.