Learning about Learning at Scale: Methodological Challenges and Recommendations

Learning at scale opens up a new frontier to learn about learning. Massive Online Open Courses (MOOCs) and similar large-scale online learning platforms give an unprecedented view of learners' behavior whilst learning. In this paper, we argue that the abundance of data that results from such platforms not only brings novel opportunities to the study of learning, but also bears novel methodological challenges. We show that the resulting data comes with various challenges with respect to the granular, observational, and large nature of these data. Additionally, we discuss a series of potential solutions, such as sharing validated models and performing pre-registered confirmatory research. With these contributions, this paper aims to increase awareness and understanding of both the strengths and challenges of research on learning at scale.