A new approach for small sample face recognition with pose variation by fusing Gabor encoding features and deep features

Small sample and pose variation are two critical technical problems in automatic face recognition (AFR). The combination of these two difficulties will seriously affect the performance of face recognition and restrict the wide application of AFR. To address this problem, we propose a new multi-feature fusion framework. First, we propose a pixel-level data augmentation algorithm based on manifold subspace partition, which constructs virtual samples in the original face image space to achieve training sample expansion and diversity enhancement. Then, we propose a feature-level data augmentation based on Gabor transformation, which can capture multi-level face features through multi-scale and multi-direction Gabor filters to realize the face expansion in feature space. To eliminate the data redundancy and interference information generated by Gabor feature augmentation, a Gabor feature encoding algorithm is proposed to construct the compressed Gabor feature vector. In addition, we propose a small scale adaptive deep CNN model, which is suitable for small sample datasets and can effectively extract nonlinear deep features of pose-varied faces. Finally, Gabor encoding features and nonlinear deep features are combined for small sample face recognition with pose variation. Experiment results based on two face datasets prove the effectiveness of the proposed multi-feature fusion framework.