A study on emotion recognition using speech acoustic features and face images

Generally, people recognize other people's emotions by their voices and facial expressions. So, speech signals and facial images have been actively studied in the field of emotional recognition. Therefore, in this paper, we present effective acoustic features for emotion recognition and a method to recognize emotions by combining speech signals and facial image sequences. To combine these the two inputs like speech signals and facial image sequences, three models are designed. And these three models are combined by using the Joint Fine Tuning method. The result shows that the performance of our model is very promising for emotion recognitions in comparison with other models using speech signals and facial image sequences. © The Korean Institute of Electrical Engineers