Sign Language Recognition of Selected Filipino Phrases Using LSTM Neural Network

The use of Filipino Sign Language (FSL) has contributed to the improvement of communication of deaf people; however, the majority of the population in the Philippines does not understand FSL. The study explored computer vision in obtaining the images and deep learning techniques in building the automated FSL recognition model to bridge the communication gap between the deaf community and the hearing majority. The model has been trained using LSTM neural network using the features extracted using MediaPipe Holistic from video files of Filipino phrases performed by three (3) FSL signers. The SLR system developed could recognize (15) continuous Filipino words. The model evaluation has shown an impressive result wherein the average accuracy achieved on the test set is 94%. In the experimentation conducted on 10 participants using the SLR system, the overall accuracy obtained on two trials is 72.38%, with an average prediction time of 0.3 s. Based on the analysis of the experiment results, the developed SLR system is robust, is time efficient, is signer independent, and can detect both manual and non-manual features of the gesture. More data will be collected for future directions to enable a conversational SLR system with more FSL vocabularies and sentences.