Video-Based Emotion Recognition in the Wild for Online Education Systems

With the rapid development of the Internet, online learning has become one of the main ways of acquiring knowledge. In order to make teachers understand students' emotional states and adjust teaching programs on time, a new video based model called the Wild Facial Spatiotemporal Network (WFSTN) is proposed in this paper for emotion recognition in online learning environments. The model consists of two modules: a pretrained DenseNet121 for extracting facial spatial features, and a Bidirectional Long-Short Term Memory (Bi-LSTM) network with self-attention for generating attentional hidden states. In addition, a dataset of student emotions in online learning environments (DSEOLE) is produced using a self-developed online educational aid system. The method is evaluated on the Acted Facial Expressions in the Wild (AFEW) and DSEOLE datasets, achieving 72.76% and 73.67% accuracy in three-class classification, respectively. The results show that the proposed method outperforms many existing works on emotion recognition for online education.