Double-handed dynamic gesture recognition using contour-based hand tracking and maximum mean probability ensembling (MMPE) for Indian Sign Language

The ability to communicate in verbal language is one of the greatest gifts of humankind. The people who do not have this ability feel isolated and struggle to convey their part in society. Sign language or gesture communication is the only method they can rely upon, but most of our community cannot understand this language without the help of a translator. The paper presents a dynamic Indian Sign Language recognition system without complicated sensors or costly devices to sense the movements of the hands. The paper proposes a problem-specific contour-based hand tracking algorithm that can track both hands simultaneously, solving the ambiguity caused by merging the hands while gesturing. The paper also proposes a maximum mean probability ensembling that combines the classification probabilities of three different classification models for better accuracy. The proposed model recognizes the double-handed dynamic gestures with an accuracy of 89.83%. The paper discusses the performance of scale-invariant feature transform, tracked image feature and their combination feature for dynamic gesture classification, and tests the discriminating power of different classifiers on these features. The support vector machine classifier showed the best performance.