Individual 3D face synthesis based on orthogonal photos and speech-driven facial animation

In the paper, a methodology for individual face synthesis using given orthogonal photos is proposed. And an integrated speech-driven facial animation system is presented. Firstly, in order to capture given subject's personal facial configuration, a novel coarse-to-fine strategy based on facial texture and deformable template is proposed to localize some facial feature points in the image of frontal view. And the corresponding feature points in the profile are extracted by using polygonal approximation. Secondly, all these feature paints are aligned to fit the generic 3D face model to a specialized one to reflect the given person's facial configuration. Then a multi-direction texture-mapping technique is presented to synthesize a lifelike personal face. Finally, muscle-based expression and lip-motion models are built up. All above technologies are integrated into a speech-driven face animation system. We are aiming at a MPEG-4 compatible video-driven face animation system.