A Robust Framework for Deep Learning Approaches to Facial Emotion Recognition and Evaluation

Facial emotion recognition (FER) is a vast and complex problem space within the domain of computer vision and thus requires a universally accepted baseline method with which to evaluate proposed models. While test datasets have served this purpose in the academic sphere, real-world application and testing of such models lacks any real comparison. Therefore, we propose a framework in which models developed for FER can be compared and contrasted against one another in a constant, standardized fashion. A lightweight convolutional neural network (CNN) is trained on the AffectNet dataset - a large, variable dataset for facial emotion recognition - and a web application is developed and deployed with our proposed framework as a proof of concept. The CNN is embedded into our application and is capable of instant, real-time facial emotion recognition. When tested on the AffectNet test set, this model achieves an accuracy of 55.09% for emotion classification of eight different emotions. Using our framework, the validity of this model and others can be properly tested by evaluating a model's efficacy not only based on its accuracy on a sample test dataset, but also on inthe-wild experiments. Additionally, our application is built with the ability to save and store any image captured or uploaded to it for emotion recognition, allowing for the curation of more quality and diverse facial emotion recognition datasets.