How Peers CommunicateWithout Words-An Exploratory Study of Hand Movements in Collaborative Learning Using Computer-Vision-Based Body Recognition Techniques

Accumulating research in embodied cognition highlights the essential role of human bodies in knowledge learning and development. Hand movement is one of the most applied body motions in the collaborative ideation task when students co-construct knowledge with and without words. However, there is a limited understanding of how students in a group use their hand movements to coordinate understandings and reach a consensus. This study explored students' hand movement patterns during the different types of knowledge co-construction discourses: quick consensus-building, integration-oriented consensus building, and conflict-oriented consensus building. Students' verbal discussion transcripts were qualitatively analyzed to identify the type of knowledge co-construction discourses. Students' hand motion was video-recorded, and their hand landmarks were detected using the machine learning tool MediaPipe. One-way ANOVA was conducted to compare students hand motions in different types of discourses. The results found there were different hand motion patterns in different types of collaboration discourses. Students tended to employ more hand motion during conflict-oriented consensus building discourses than during quick consensus building and integration-oriented consensus building discourses. At the group level, the collaborating students were found to present less equal hand movement during quick consensus-building than integration-oriented consensus building and conflict-oriented consensus building. The findings expand the existing understanding of embodied collaborative learning, providing insights for optimizing collaborative learning activities incorporating both verbal and non-verbal language.