Don't Read My Lips: Assessing Listening and Speaking Skills Through Play with a Humanoid Robot

This study investigates the potential of using the humanoid robot, NAO, as a playful tool for assessing the listening and speaking skills of seven hearing-impaired students who use cochlear implant(s) and sign language as their main communication modality. NAO does not have a human mouth and therefore, students cannot do lip-reading; we considered this to be a unique characteristic of the technology that can help make the assessment of listening and speaking skills efficient and accurate. Three game-like applications were designed and deployed on NAO for the purpose of this study. Results demonstrated how NAO was successfully used in this context. Our results, although preliminary, should encourage future research in the area of listening and speaking assessment for hearing impaired children, as well as speech enhancement via play with social robots.