A Modern Approach for Sign Language Interpretation Using Convolutional Neural Network

There are nearly 70 million deaf people in the world. A significant portion of them and their families use sign language as a medium for communicating with each other. As automation is being gradually introduced to many parts of everyday life, the ability for machines to understand the act on sign language will be critical to creating an inclusive society. This paper presents multiple convolutional neural network based approaches, suitable for fast classification of hand sign characters. We propose two custom convolutional neural network (CNN) based architectures which are able to generalize 24 static American Sign Language (ASL) signs using only convolutional and fully connected layers. We compare these networks with transfer learning based approaches, where multiple pre-trained models were utilized. Our models have remarkably outperformed all the preceding models by accomplishing 86.52% and 85.88% accuracy on RGB images of the ASL Finger Spelling dataset.