In-Vehicle Hand Gesture Recognition using Hidden Markov Models

In this work we explore Hidden Markov models as an approach for modeling and recognizing dynamic hand gestures for the interface of in-vehicle infotainment systems. We train the HMMs on more complex shape descriptors such as HOG and CNN features, unlike typical HMM based approaches. An analysis of the optimal hyperparameters of the HMM for the task has been carried out. Also, dimensionality reduction and data augmentation have been explored as methods for reducing overfitting of the HMMs. Finally we experiment with the CNN-HMM hybrid framework which uses a trained Convolutional Neural Network for estimating the emission probabilities of the HMM. We obtain a mean recognition accuracy of 57.50% on the VIVA hand gesture challenge, which while not the best result on the dataset, shows the feasibility of the approach.