FACE MASK AWARE ROBUST FACIAL EXPRESSION RECOGNITION DURING THE COVID-19 PANDEMIC

Wearing face masks is considered an effective means of preventing the transmission of coronavirus during the COVID-19 pandemic. Facial expression recognition (FER) under partial occlusion, especially with face masks, makes it a challenging task in the research area of computer vision. In this paper, we propose a two-stage attention model to improve the accuracy of face-mask-aware FER: In stage 1, we train the masked/unmasked binary deep classifier, which can generate attention heatmaps to roughly distinguish the masked facial parts from the unobstructed region. In stage 2, we train the FER classifier, which is guided to pay more attention to the region that is essential to the facial expression classification, and both occluded and non-occluded regions are taken into consideration but reweighed. The proposed method outperforms other state-of-the-art occlusion-aware FER methods on face-mask-aware FER datasets, whether in the wild or in the laboratory.