Joint Bayes filter: A hybrid tracker for non-rigid hand motion recognition

In sign-language or gesture recognition, articulated hand motion tracking is usually a prerequisite to behaviour understanding. However the difficulties such as non-rigidity of the hand, complex background scenes, and occlusion etc make tracking a challenging task. In this paper we present a hybrid HMM/Particle filter tracker for simultaneously tracking and recognition of non-rigid hand motion. By utilising separate image cues, we decompose complex motion into two independent (non-rigid/rigid) components. A generative model is used to explore the intrinsic patterns of the hand articulation. Non-linear dynamics of the articulation such as fast appearance deformation can therefore be tracked without resorting to a complex kinematic model. The rigid motion component is approximated as the motion of a planar region, where a standard particle filter method suffice. The novel contribution of the paper is that we unify the independent treatments of non-rigid motion and rigid motion into a robust Bayesian framework. The efficacy of this method is demonstrated by performing successful tracking in the presence of significant occlusion clutter.