Detecting Affect States Using VGG16, ResNet50 and SE-ResNet50 Networks

Affect detection is a key component in developing intelligent human computer interface systems. State-of-the-art affect detection systems assume the availability of full un-occluded face images. This work uses convolutional neural networks with transfer learning to detect 7 basic affect states, viz. Angry, Contempt, Disgust, Fear, Happy and Sad. The paper compares three pre-trained networks, viz. VGG16, ResNet50 and a SE-ResNet50, in which a new architectural block of squeeze and excitation has been integrated with ResNet50. Modified VGG-16, ResNet50 and SE-ResNet50 networks are trained on images from the dataset, and the results are compared. We have been able to achieve validation accuracies of 96.8%, 99.47%, and 97.34% for VGG16, ResNet50 and SE-ResNet50, respectively. Apart from accuracy, the other performance matrices used in this work are precision and recall. Our evaluation, based on these performance matrices, shows that accurate affect detection is obtained from all the three networks with Resnet50 being the most accurate. © 2020, Springer Nature Singapore Pte Ltd.