Face detection and tracking in video sequences using the modified census transformation

We present the combination of an illumination invariant approach to face detection combined with a tracking mechanism used for improving speed and accuracy of the system. We introduce illumination invariant local structure features for object detection. For an efficient computation we propose a modified census transform, which enhances the original work of Zabih and Woodfill [19] [Ramin Zabih, John Woodfill. A non-parametric approach to visual correspondence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1996.]. The tracking is performed by means of continuous detection. We show that the advent of new rapid detection algorithms may change the need for traditional tracking. Furthermore the mentioned problems have a natural solution within the presented tracking by continuous detection approach. The only assumption on the object to track is its maximal speed in the image plane, which can be set very generously. From this assumption we derive three conditions for a valid state sequence in time. To estimate the optimal state of a tracked face from the detection results a Kalman filter is used. This leads to an instant smoothing of the face trajectory. It can be shown experimentally that smoothing the face trajectories leads to a significant reduction of false detections compared to the static detector without the presented tracking extension. We further show how to exploit the highly redundant information in a natural video sequence to speed-up the execution of the static detector by a temporal scanning procedure which we call, 'slicing'. A demo program showing the outcomes of our work can be found in the inter-net under http://www.iis.fraunhofer.de/bv/biometrie/ for download. (c) 2005 Elsevier B.V. All rights reserved.