2D and 3D multimodal hybrid face recognition

We present a 2D and 3D multimodal hybrid face recognition algorithm and demonstrate its performance on the FRGC v1.0 data. We use hybrid (feature-based and holistic) matching for the 3D faces and a holistic matching approach on the 2D faces. Feature-based matching is performed by offline segmenting each 3D face in the gallery into three regions, namely the eyes-forehead, the nose and the cheeks. The cheeks are discarded to avoid facial expressions and hair. During recognition, each feature in the gallery is automatically matched, using a modified ICP algorithm, with a complete probe face. The holistic 3D and 2D face matching is performed using PCA. Individual matching scores are fused after normalization and the results are compared to the BEE baseline performances in order to provide some answers to the first three conjectures of the FRGC. Our multimodal hybrid algorithm substantially outperformed others by achieving 100% verification rate at 0.0006 FAR.