A Real-Time Hand Pose Recognition Method with Hidden Finger Prediction

In this paper, we present a real-time hand pose recognition method to provide an intuitive user interface through hand poses or gestures without a keyboard and a mouse. For this, the areas of right and left hands are segmented from the depth camera image, and noise compensation is performed. Then, the rotation angle and the centroid point of each hand area are calculated. Subsequently, joint points and end points of a finger are detected by expanding a circle at regular intervals from a centroid point of the hand. Lastly, the hand pose is recognized by matching between the current hand information and the hand model of previous frame and the hand model is updated for the next frame. This method enables users to predict the hidden fingers through the hand model information of the previous frame using temporal coherence in consecutive frames. As a result of the experiment on various hand poses with the hidden fingers using both hands, the accuracy showed over 95% and the performance indicated over 32 fps. The proposed method can be used as a contactless input interface in presentation, advertisement, education, and game applications.