A deep sign language recognition system for Indian sign language

Deaf people face major challenges during communication with normal people. Employing a human interpreter (a person who converts Sign language (SL) into a language that the normal/hearing community can understand) is not an effective solution to this problem due to the unavailability of professional interpreters. Thus, the Sign Language Recognition System (SLRS) is the most efficient and effective choice because it automatically converts SL into text/speech without an interpreter and reduces the communication barrier between deaf and normal people. This paper reports a work on Indian Sign Language (ISL) word recognition using a vision-based technique. The existing vision-based solutions for ISL word recognition are ineffective due to excessive pre-processing such as extracting features from a large sequence of frames. Therefore, a vision-based SLRS named Hybrid CNN-BiLSTM SLR (HCBSLR) is proposed, which overcomes the drawback of excessive pre-processing. The proposed model uses a Histogram Difference (HD) based key-frame extraction method to improve the accuracy and efficiency of the system by eliminating redundant or useless frames. The HCBSLR system uses VGG-19 for spatial feature extraction and Bidirectional Long Short Term Memory (BiLSTM) for temporal feature extraction. The proposed HCBSLR system has achieved an average accuracy of 87.67%, which is compared with some of the existing SLRS. The experimental results show that the proposed HCBSLR system is more accurate and efficient than the existing SLRS.