Evaluating a simulated student using real students data for training and testing

SimStudent is a machine-learning agent that learns cognitive skills by demonstration. It was originally developed as a building block of the Cognitive Tutor Authoring Tools (CTAT), so that the authors do not have to build a cognitive model by hand, but instead simply demonstrate solutions for SimStudent to automatically generate a cognitive model. The SimStudent technology could then be used to model human students' performance as well. To evaluate the applicability of SimStudent as a tool for modeling real students, we applied SimStudent to a genuine learning log gathered from classroom experiments with the Algebra I Cognitive Tutor. Such data can be seen as the human students' "demonstrations" of how to solve problems. The results from an empirical study show that SimStudent can indeed model human students' performance. After training on 20 problems solved by a group of human students, a cognitive model generated by SimStudent explained 82% of the problem-solving steps performed correctly by another group of human students.