Deep Learning Based Gesture Classification for Hand Physical Therapy Interactive Program

In this paper, we propose using the Google Colab deep learning framework to create and train convolutional neural networks from scratch. The trained network is part of a core artificial intelligent feature of our interactive software game, aiming to encourage white-collar workers to exercise hands and wrists frequently through playing the game. At this moment, the network is trained with our self-collected dataset of 12,000 bare-hand gesture images shot against a static dark background. The network focuses on classifying a still image into one of the six predefined classes of gestures and it seems to cope well with slight variation in size, skin tone, position and orientation of hand. This network is designed to be light in computation with real-time running time even on CPU. The network yields 99.68% accuracy on the validation set and 78% average accuracy when being tested with 50 different users. Our experiment on actual users reveals useful insight about problems using a deep learning based classifier in a real-time interactive system.