Regional-CNN-based enhanced Turkish sign language recognition

Hearing impaired people use sign language for communication. Considering the spoken languages, the most widely spread ones have their own sign languages. Thus, sign languages are spoken language dependent. In this paper, the focus is on Turkish Sign Language (TSL). In parallel with the developments in computer science and artificial intelligence, the human computer interaction is now possible and it is a definite field of computer science. Automatically detecting and recognizing sign languages by computers are also possible applications. The improvements in machine learning field in the last decade made it possible to improve systems that can automatically detect and recognize sign language from still images. On the other hand, using a robust algorithm that is recognizing a sign language may not be applicable to another language as signs and linguistic properties may differ between languages. In this paper, we propose a novel method of recognizing characters of Turkish Sign Language (TSL). The method is tested on captured images containing signs, which are extracted from video files. Alexnet is employed as a pre-trained network in this system. Region-based Convolutional Neural Network (R-CNN) object detector is used to train the new model. The knowledge of the neural network is transferred using transfer learning method and tuned for recognizing TSL. The system achieves 99.7% average precision that is acceptable and comparable to conventional methods applied on TSL. Furthermore, the study is significant since it is one of the first successful application of transfer learning approach to TSL. The system is open to further improvements by improving the representations of the sign images.