Investigating Factors Influencing Students' Assessment of Conceptual Models

This paper discusses the challenges in evaluating the quality of conceptual models in educational settings. While automated grading techniques may work for simplistic modeling tasks, realistic modeling tasks that allow for a wide variety of solutions cannot be evaluated using automated techniques. However, the traditional approach of having instructors grade the exercises may not be feasible in larger courses. To address this issue, alternative approaches, such as educating students to assess the quality of their own solutions or using calibrated peer reviews, can be used. Therefore, it is crucial to identify the quality of feedback a student can deliver on their own. As a first step, this paper reports on the results of controlled experiments with 368 participants to investigate factors that influence students' model comprehension and to identify ways to distinguish good student assessments from bad ones.