Multimodal face shape detection based on human temperament with hybrid feature fusion and Inception V3 extraction model

In this research, the multi model (image and video) based face shape detection with human temperament is developed. Here, the video is captured by webcam via live recording session and the proposed model undergoes three major stages namely, pre-processing, extracted feature fusion and temperament detection. Extraction of facial landmark and facial boundary takes place in the pre-processing stage. In the feature extraction stage, the handcrafted features from image are extracted (i.e. face, forehead, eyes, cheeks, nose and mouth). From the video frames, the intrinsic features related to face region are extracted using pretrained Inception V3 model. Then robust principal component analysis (RPCA) is introduced to reduce the dimension of extracted features. Further, the feature fusion process is performed using discriminant correlation analysis (DCA) and canonical correlation analysis (CCA) at hybrid phase. Finally, the gated recurrent unit (GRU) classifier model is applied to identify the human temperaments based on face shapes. In the experimental scenario, the performance measures of accuracy (98.51%, 98.86%), precision (96.14%, 97.89%), recall (96.34%, 97.95%), F-measures (96.24%, 97.94%), etc are evaluated and compared with state-of-the-art methods under two datasets. In addition to this, the statistical test is also conducted to validate the efficacy of the proposed model.