Human-Machine Interaction based on Hand Gesture Recognition using Skeleton Information of Kinect Sensor

The hand gesture provides a natural and intuitive communication medium for the human and machine interaction. Because, it can use in virtual reality, language detection, computer games, and other human-computer or human-machine instruction applications. Currently, the sensor and camera-based application is a field of interest for many researchers. This paper proposes a new hand gesture recognition system using the Kinect sensor's skeleton data, which works in an environment where people do not touch devices or communicate verbally. The proposed model focuses on mainly two modules, namely, hand area and fingertip detection, and hand gesture recognition. The hand area and fingertip are detected by positioning the palm point and find extreme of contour. And, the hand gesture is recognized by measuring the distance between different body indexes of skeleton information. Here, six gestures instructions are considered such as move right to left, move left to right, move up to down, move down to up, open and closed, and also recognize the numeric number using the fingertip. This system is able to detect the presence of hand area and fingers and to recognize different hand gestures. As a result, the average recognition accuracy of different hand gestures and stretched fingers numbers are 95.91% and 96%, respectively.