A convolutional neural network and classical moments-based feature fusion model for gesture recognition

Hand gesture recognition is a significant and challenging building block for different computer vision applications such as controlling, conversational, manipulative, and communicative gestures. Several systems have been suggested to address the hand gesture recognition and classification challenges. Convolutional neural networks (CNNs) are widely used for different pattern recognition problems. Besides CNNs, the features extracted using moment-based approaches are considered the most effective and transparent features for the task of image recognition and classification. However, most of the moment-based approaches consider only the global image features while neglecting the discriminative properties of the local image features. This paper proposes a new efficient gesture recognition approach that combines CNN features with conventional Zernike moment-based features. Two groups of Zernike moment-based features are extracted since only global Zernike moment-based features are not sufficient to distinguish between very similar hand postures. Hence, global features are supplemented with local modified Zernike moment-based features to improve the recognition accuracy by extracting the local pattern information of the image. Furthermore, we have introduced an improved architecture that combines the features derived from the whitening transformed Zernike moments computed for each image and CNNs' last convolutional layer. Finally, the library for support vector machine (LIBSVM) has been used for classification. The proposed model has recognition accuracies of 98.41%, 94.33%, 97.27%, and 99.84% on four different standard datasets. The performance comparisons show that the proposed model is better than the state-of-the-art methods.