Automatic Multi-view Face Recognition via 3D Model Based Pose Regularization

One of the major challenges encountered by face recognition lies in the difficulty of handling arbitrary poses variations. While different approaches have been developed for face recognition across pose variations, many methods either require manual landmark annotations or assume the face poses to be known. These constraints prevent many face recognition systems from working automatically. In this paper, we propose a fully automatic method for multi-view face recognition. We first build a 3D model from each frontal target face image, which is used to generate synthetic target face images. The pose of a query face image is also estimated using a multi-view face detector so that the synthetic target face images can be generated to resemble the pose variation of a query face image. Procrustes analysis is then applied to align the synthetic target images and the query image, and block based MLBP features are extracted for face matching. Experimental results on two public-domain databases (Color FERET and PubFig), and a Mobile face database collected using mobile phones show that the proposed approach outperforms two state-of-the-art face matchers (FaceVACS and MKD-SRC) in automatic multi-view face recognition. The proposed approach can also be easily extended to leverage existing face recognition systems for automatic multi-view face recognition.