A visual system for hand gesture recognition in human-computer interaction

Visual hand gestures offer an interesting modality! for Human-Computer-Interaction (HCI) applications. Gesture recognition and hand tracking, however, are not trivial tasks and real environments set a lot of challenges to algorithms performing such activities. In this paper, a novel combination of techniques is presented for tracking and recognition of hand gestures in real, cluttered environments. In addition to combining existing techniques, a method for locating a hand and segmenting it from an arm in binary silhouettes and a foreground model for color segmentation is proposed. A single hand is tracked with a single camera and the trajectory information is extracted along with recognition of five different gestures. This information is exploited for replacing the operations of a normal computer mouse. The silhouette of the hand is extracted as a combination of different segmentation methods: An adaptive colour model based segmentation is combined with intensity and chromaticity based background subtraction techniques to achieve robust performance in cluttered scenes. An affine-invariant Fourier-descriptor is derived from the silhouette, which is then classified to a hand shape class with support vector machines (SVM). Gestures are recognized as changes in the hand shape with a finite state machine (FSM).