A Hybrid Facial Expression Recognition System Based on Recurrent Neural Network

Facial expression recognition (FER) is an important and challenging problem for automatic inspection of surveillance videos. In recent years, with the progress of hardware and the evolution of deep learning technology, it is possible to change the way of tackling facial expression recognition. In this paper, we propose a sequence-based facial expression recognition framework for differentiating facial expression. The proposed framework is extended to a frame-to-sequence approach by exploiting temporal information with gated recurrent units. In addition, facial landmark points and facial action unit are also used as input features to train our network which can represent facial regions and its components effectively. Based on this, we build a robust facial expression system and is evaluated using two publicly available databases. The experimental results show that despite the uncontrolled factors in the videos, the proposed deep learning-based solution is consistent in achieving promising performance compared to that of the former schemes.