Video-Based Facial Expression Recognition Using a Deep Learning Approach

This research aims at classifying facial expression of humans in a video. Facial expressions were classified into one of the following common facial expression classes that are anger, disgust, fear, happiness, sadness, surprise, and neutral. To accomplish this task, convolutional neural networks were developed, to classify each facial image extracted from a frame into one of the seven classes of facial expressions we have chosen. The model was developed in Keras and tensorflow. Frames were extracted using OpenCV to detect location of facial image from each frame. Face detector was used based on SSD framework (single-shot multi-box detector), using a reduced ResNet-10 model. On all the facial images detected, their expressions were classified using the developed CNN model and based on results of entire images, a table is prepared to classify in which expression has been identified most in the video. Finally, we compare the results of frame extracted at two different rates, i.e., 1 fps and 10 fps.