A Spatio-Temporal Framework for Dynamic Indian Sign Language Recognition

A sign language recognition system is a boon to the signer community as it eases the flow of information between the signer and non-signer communities. However, extracting timely detail from the video data is still a challenging task. In this paper, a deep learning based model consisting of trainable CNN and trainable stacked 2 bidirectional long short term memory (S2B-LSTM) has been proposed and tested to recognise the dynamic gestures of Indian sign language (ISL). The CNN architecture has been used as feature extractor to extract the spatial features from the input video data, whereas the temporal relation between the consecutive frames of input video is extracted using S2B-LSTM. This model has been trained and tested on self-developed dataset consisting of 360 videos of ISL dynamic gestures. The CNN-S2B-LSTM model outperforms the existing techniques of sign language recognition with best recognition accuracy of 97.6%.