Tracking a face for knowledge-based coding of videophone sequences

Tracking a face is one of the important topics for knowledge-based coding of videophone sequences and also for the representation of 3D objects within MPEG-4 Synthetic/Natural Hybrid Coding (SNHC). Up to now, the face model has been tracked by global head motion compensation. Because the 3D head model shape affects the accuracy of motion estimation, an inaccurate head model shape reduces the accuracy of face tracking. In this paper, a new algorithm for tracking a face combining global head motion compensation and the update of the face model Candide during the sequence is proposed. As a first stage of the proposed algorithm, face tracking only by global head motion compensation is used. After that, the 2D center positions of the eyes and the mouth of a person in the image sequence are estimated using template matching and feature point extraction techniques. Then, the shape of the face model Candide is updated during the sequence using these estimated 2D center positions. This proposed algorithm has been applied to typical videophone sequences with a spatial resolution corresponding to CIF and a frame rate of 10 Hz. For evaluation, error criteria have been introduced which give position errors of the eyes and the mouth averaged over a whole sequence. The experimental results show that the proposed algorithm reduces the average position errors for the eyes and the mouth by 48% and 53%, respectively, compared to face tracking by global head motion compensation only. (C) 1997 Elsevier Science B.V.