A meta-analysis on the effects of technology's functions and roles on students' mathematics achievement in K-12 classrooms

Background With the recent pivot to online instruction and/or to mixed online/face-to-face (i.e., hybrid) models of teaching necessitated by the novel coronavirus pandemic, the uses of technology to support instruction would seem to have great importance. Whereas effective integration of technology in mathematics classrooms requires recognition of students' needs and instructional goals that technology could fulfill, very few comprehensive meta-analytical studies have focused on the roles of instructional technology in support of specific learning goals. Objectives In this meta-analysis, we investigated and reviewed the roles and affordances of technology in mathematics instruction and its effects on students' mathematics achievement. Methods Cross-searching multiple bibliographic databases generated a total of 3276 potentially relevant full-text studies. After applying a series of inclusion/exclusion criteria, we kept 77 studies with 151 effect sizes. Two researchers then coded for seven sets of study features and student characteristics: technology roles, publication-related features (years and resources), research design, sample size, grade levels, duration of interventions, and types of the testing instruments. The coding achieved acceptable levels of inter-rater reliabilities. Since effect sizes varied across studies and student populations, random-effects models were used for overall effect estimates and moderator analyses. Results and conclusions Overall, this meta-analysis revealed a small, positive, and statistically significant effect of technology on students' mathematics achievement with an average Hedges' s g (g ($) over bar ) of 0.23. The largest significant moderator effects were found when technology was used to design and support collaborative and communicative environments (g ($) over bar = 0.49); the next-largest moderator effects were found when using technology as supports for problem-solving (g ($) over bar = 0.39), followed by the roles of as supports for conceptual development (g ($) over bar = 0.31) and as adaptive mathematics practices (g ($) over bar = 0.28). This study also found that the effects of technology were differentiated as a function of publication resources, research design, duration of intervention, and the testing instrument. Major takeaways Focusing on the roles and functions of technology, this study can contribute to guide potential synthetical studies as well as the future design of educational technology. In particular, technology is used effectively when it is used (a) to create and design a collaborative and communicative learning environment where students have chances to collaborate and interact with each other and (b) as supports for mathematics problem-solving and for mathematics conceptual development. On the other hand, technology is less effective when it is used for monitoring and assessing students' learning processes without follow-up instructions. The findings suggested that technology intervention lasting for a shorter time is the better choice. When technology is used for longer periods of instruction, maintaining its novelty and students' curiosity should be considered. This study also suggested that incorporating computer-adaptive testing in technology-based intervention programs is more likely to detect technology effects as adaptive testing could continuously adjust to the abilities of individual students.