Class-Balanced and Local Median Loss Jointly Supervised for Wild Facial Expression Recognition

Over the past few years, convolutional neural networks have shown effective performance on laboratory-controlled facial expression recognition. However, it is still a challenge problem for facial expression recognition in the wild. In this paper, a loss function - CALM loss (class-balanced and local median) is proposed to solve the problem of imbalanced data for wild facial expression recognition and large intra-class variation caused by posture, illumination and gender. The CALM loss includes two parts: the class-balanced Softmax loss function and the local median loss function. The class-balanced Softmax loss function marks the two expressions of fear and disgust, which have a small amount of data and are prone to misclassification, as difficult samples, and the other five expressions as easy samples. During the network training, the weight of difficult samples is adaptively increased to improve the recognition accuracy of difficult samples, so as to improve the average accuracy of expression recognition. In addition, there are some samples in each category that are far away from the majority of the samples in the category, and their existence will cause the center of the category calculated by the mean method to deviate from the majority of the samples in the category. The local median loss function uses the median value of several neighbors that belong to the same category as each sample as the class center, which can reduce the impact of outlier samples on the choice of category center to a certain extent. The average recognition accuracy on the RAF (real-world affective faces) dataset was improved by 1.32% compared with local subclass method, which proves the effectiveness of the proposed method. © 2020, Beijing China Science Journal Publishing Co. Ltd. All right reserved.