Confounded or Controlled? A Systematic Review of Media Comparison Studies Involving Immersive Virtual Reality for STEM Education

A substantial amount of media comparison research has been conducted in the last decade to investigate whether students learn Science, Technology, Engineering, and Mathematics (STEM) content better in immersive virtual reality (IVR) or more traditional learning environments. However, a thorough review of the design and implementation of conventional and IVR conditions in media comparison studies has not been conducted to examine the extent to which specific affordances of IVR can be pinpointed as the causal factor in enhancing learning. The present review filled this gap in the literature by examining the degree to which conventional and IVR conditions have been controlled on instructional methods and content within the K-12 and higher education STEM literature base. Thirty-eight published journal articles, conference proceedings, and dissertations related to IVR comparison studies in STEM education between the years 2013 and 2022 were coded according to 15 categories. These categories allowed for the extraction of information on the instructional methods and content characteristics of the conventional and IVR conditions to determine the degree of control within each experimental comparison. Results indicated only 26% of all comparisons examined between an IVR and conventional condition were fully controlled on five key control criteria. Moreover, 40% of the comparisons had at least one confound related to instructional method and content. When looking at the outcomes of the studies, it was difficult to gather a clear picture of the benefits or pitfalls of IVR when much of the literature was confounded and/or lacked sufficient information to determine if the conditions were controlled on key variables. Implications and recommendations for future IVR comparison research are discussed.