Video-based face motion measurement

In this paper, we describe and evaluate a noninvasive method of measuring face motion during speech production. Reliable measures are extracted from standard video sequences using an image analysis process that takes advantage of important constrainst on face structure and motion. Measures are made by deforming the surface of an ellipsoidal mesh fit to the face image at successive degrees of spatial resolution using a two-dimentional wavelet transform of the image frames. Reliability of the measures is evaluated through comparison with 3D marker data recorded for the same utterances and speaker, and perceptual evaluation of the talking head animations created from the measures are underway. © 2002 Published by Elsevier Science Ltd.