An Intelligent LEGO Tutoring System for Children with Autism Spectrum Disorder

Prior studies have shown the effectiveness of LEGO-based activities to improve children.s social skills, especially those with autism spectrum disorder (ASD). In such activities, teachers are expected to provide instructions/guidance and facilitate communication among children during their collaborative play. When the ratio of children to teacher increases, an intelligent monitoring system could be very helpful to prompt the teachers of any necessary intervention, or to provide children some trivial feedback so that their teachers could focus on other tasks. In this paper, we propose such a system to assist both teachers and children in playing with LEGO bricks. At first, teachers may use the system to create some models. The system will then generate step-by-step instructions in both visual and auditory forms. In addition, the system can track the children.s brick-building process in real time and provide some feedback when any mistakes were made. Two cameras with a unique image recognition module are used to capture the 3D structure of LEGO bricks.