CC-CNN: A cross connected convolutional neural network using feature level fusion for facial expression recognition

Facial expressions are an important form of non-verbal communication as they directly reflect the internal emotions of a person. The primary task of automated Facial Expression Recognition (FER) systems lies in extracting salient features related to facial expressions. In this paper, a Cross Connected Convolutional Neural Network (CC-CNN) has been proposed for extracting the facial features. The proposed CC-CNN model contains two levels of input for extracting the features related to facial expressions. Cyclopentane Feature Descriptor (CyFD), inspired by cyclopentane’s structure, has been proposed to extract significant features. The feature response map generated by the CyFD method has been given as input to the first level, and in the second level, the features have been extracted directly from the facial image. The input images from both levels are passed through a series of convolutional layers with cross connections for extracting the fused (local and global) features related to facial expressions. Finally, towards the end, the CC-CNN method works by fusing all the features extracted from both levels. To validate the efficiency of the proposed CC-CNN method, the experiments have been performed on benchmark FER datasets such as CK+, MUG, RAF, FER2013 and FERG. The comparison results from the experimental analysis revealed that the proposed model outperformed the recent FER methods.