An Adaptive Thresholding-Based Movement Epenthesis Detection Technique Using Hybrid Feature Set for Continuous Fingerspelling Recognition

Sign language recognition systems are gaining importance in recent times as these have established themselves as important elements of human–computer interaction. Also these provide an opportunity for the deaf and hearing impaired to com-municate with the common people without the need of an interpreter. Yet there are plenty of challenges in this field which are worth exploring and solutions formulated. In this paper, we have addressed the design of a continuous fingerspelling recognition system which segments a fingerspelling sequence into meaningful extracts and non-sign patterns and thereby recognizes the meaningful signs. Sign segmentation is carried out by means of a unique set of features which comprises of shape matching, velocity change and displacement of centroid between successive frames. Specialized techniques like adaptive thresholding and finite state machine model are also incorporated into our system for efficient classification of sign and movement epenthesis frames. We have validated the performance of our proposed system taking into account the continuous fingerspelling alphabets of American Sign Language considering both native and non-native signers and have obtained an accuracy of almost 91.29%. Our proposed system also has the potential to tackle complex backgrounds involving multiple objects, backgrounds with multiple signers and different brightness conditions. © The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd 2021.