Effects of real-time adaptivity of scaffolding: Supporting pre-service mathematics teachers' assessment skills in simulations

Background: Scaffolding pre-service teachers' assessment process in video-based simulations can enhance their acquisition and refinement of assessment skills, for example, needed for accurate judgments of students' mathematical proof skills. Adapting this scaffolding to learners' individual learning processes, for example, based on text data during the assessment process, brings potential for increased learning gains. Aims: In this study, we investigated the effectiveness of adaptive scaffolding based on real-time process data, specifically targeting pre-service mathematics teachers' assessment skills regarding students' mathematical proof skills in geometry. Sample: Participants were 245 pre-service teachers. Methods: In a pre- and post-test, participants completed a video-based simulation to measure their assessment skills regarding students' mathematical proof skills. During the intervention, participants were randomly assigned to complete the video-based simulation (i) without scaffolding, (ii) with non-adaptive scaffolding, or (iii) with adaptive scaffolding. Results: We did not find significant benefits of adaptive scaffolding in enhancing pre-service teachers' judgment accuracy, aligning with prior research. For an in-depth analysis, we developed and applied a scheme to systematically validate design decisions for adaptive support. This scheme focuses on the selection and measurement of the source of adaptation and the employed support mechanisms. Applying this scheme pointed towards effects of adaptive scaffolding during the assessment process. Conclusions: This study highlights the need for proximal measures to describe learning in short interventions, explores the intricacies of adaptive scaffolding, such as overlapping with design-loop adaptivity or the accuracy of automated coding, and provides a scheme for an in-depth evaluation of the adaptivity of scaffolding.