Multimodal Facial Expression Recognition Based on 3D Face Reconstruction from 2D Images

In this paper, a novel feature extraction method was proposed for facial expression recognition. A 3D Facial Expression Generic Elastic Model (3D FE-GEM) was proposed to reconstruct an expression-invariant 3D model of each human face in the present database using only a single 2D frontal image with/without facial expressions. Then, the texture and depth of the face were extracted from the reconstructed model. Afterwards, the Gabor filter bank was applied to both texture and reconstructed depth of the face to extract the feature vectors from both texture and reconstructed depth images. Finally, by combining 2D and 3D feature vectors, the final feature vectors are generated and classified by the Support Vector Machine (SVM). Favorable outcomes were acquired for facial expression recognition on the available image database based on the proposed method compared to several state-of-the-arts in facial expression recognition.