Occlusion-adaptive fusion for gait-based motion recognition

This paper presents a new architecture for motion- and video-based event recognition using the fusion of multiple hidden Markov models (HMM) with a Bayesian belief network. We begin with a fifteen parameter structural model of the human body, where unique parameter groups and extracted gait variables define individual nodes of the network. Each node is characterized by a conditional probability mass function (PMF) in addition to evidence regarding its current state given a set of observations. The evidence for each state is virtual and derived from the conditional output probabilities of two HMMs; one represents a fundamental activity while the other defines a tracking failure event. This novel integration provides a means of recognizing a variety of activities in the presence of noise, occlusion, ambiguity, and entirely missing observations. We demonstrate the effectiveness of our approach on numerous multi-view video sequences of complex human motion.