Facial Emotion Recognition in Continuous Video

Facial emotion recognition the detection of emotion states from video of facial expressions has applications in video games, medicine, and affective computing. While there have been many advances, an approach has yet to be revealed that performs well on the non-trivial Audio/Visual Emotion Challenge 2011 data set. A majority of approaches still employ single frame classification, or temporally aggregate features. We assert that in unconstrained emotion video, a better classification strategy should model the change in features, versus simply combining them We compute a derivative of features with histogram dtfferencing and derivative of Gaussians and model the changes with a hidden Markov model. We are the first to incorporate temporal information in terms of derivatives. The efficacy of the approach is tested on the non-trivial AVEC20.1.1 data set and increases classification rates on the data by as much as 13%.