Pose invariant face recognition under arbitrary illumination based on 3D face reconstruction

Pose and illumination changes from picture to picture are two main barriers toward full automatic face recognition. In this paper, a novel method to handle both pose and lighting condition simultaneously is proposed, which calibrates the pose and lighting condition to a pre-set reference condition through an illumination invariant 3D face reconstruction. First, some located facial landmarks and a priori statistical deformable 3D model are used to recover an elaborate 3D shape. Based on the recovered 3D shape, the "texture image" calibrated to a standard illumination is generated by spherical harmonics ratio image and finally the illumination independent 3D face is reconstructed completely. The proposed method combines the strength of statistical deformable model to describe the shape information and the compact representations of the illumination in spherical frequency space, and handle both the pose and illumination variation simultaneously. This algorithm can be used to synthesize virtual views of a given face image and enhance the performance of face recognition. The experimental results on CMU PIE database show that this method can significantly improve the accuracy of the existed face recognition method when pose and illumination are inconsistent between gallery and probe sets.