Trajectory Image based Dynamic Gesture Recognition with Convolutional Neural Networks

Robust dynamic gesture recognition algorithm is of great value for kinds of intelligent interactive systems. Most current researches on this field are based on trajectory time-series, which is unstable and limited. In this paper, we proposed a novel method to realize dynamic gesture recognition by analyzing the static trajectory images with Convolutional Neural Networks (CNN). First of all, a new motion-capture device named Leap Motion is used to track fingertip positions. An effective gesture spotting algorithm is applied to identify the start/end points of dynamic gestures. Then, we map the 3D fingertip coordinates to an image acquisition window frame by frame to get the corresponding trajectory images. After a series of preprocessing steps, the normalized trajectory images are fed to a CNN model. We test the performance of the proposed method on a self-built database, and experimental results show the effectiveness for dynamic gestures recognition of numbers 0-9, with the average recognition rate up to 98.8%.