Emotional State of Children with ASD and Intellectual Disabilities: Perceptual Experiment and Automatic Recognition by Video, Audio and Text Modalities

The paper presents the results of perceptual experiments (by humans) and automatic recognition of the emotional states of children with Autism Spectrum Disorders (ASD) and Intellectual Disabilities (ID) by video, audio and text modalities. The participants of the study were 50 children aged 5 - 15 years: 25 children with ASD, 25 children with ID, and 20 adults - the participants of the perceptual experiment. Automatic analysis of facial expression by video was performed using FaceReader software runs on the Microsoft Azure cloud platform and convolutional neural network. Automatic recognition of the emotional states of children by speech was carried out using a recurrent neural network. This study was conducted in accordance with the design developed in the study of the recognition of the emotional states of children with Down syndrome by facial expression, voice, and text. The results of the perceptual experiment showed a greater accuracy in recognizing the emotional states of children with ASD and ID in comparison with automatic classification. The emotions of children with ASD are more accurately recognized by the video modality, children with ID by the text modality. The novelty of the research is the comparative results for groups of children with similar and overlapping symptoms of ASD and ID, and in setting tasks related to the analysis of the emotional sphere of children with atypical development.