Multimodal Face Recognition for Profile Views Based on SIFT and LBP

This paper presents a multimodal face recognition system for profile views in such scenarios as frontal face images are not available. According to the special physiological location relationship and their supplement between face and ear, a framework of multimodal recognition system fusing profile-view face classifier and ear classifier is proposed. In profile-view face classifier, a new feature extraction and recognition algorithm based on Scale Invariant Feature Transform(SIFT) and local binary patterns (LBP) is applied. Firstly, a set of keypoints are extracted from the face image for matching by applying the SIFT algorithm; Secondly, each keypoint is described by the rotation-invariant LBP patterns; Finally, the matching pairs between the two sets of keypoints are determined by using the nearest neighbor distance ratio based matching strategy. Ear classifier is set up based on LDA. Then decision fusion of multimodal face recognition is carried out using the combination methods of Product, Sum, Median and Vote rules according to the Bayesian theory. The results of experiment show that our method improves the recognition performance and provides a new approach of non-intrusive recognition.