Multilevel Learner Modeling in Training Environments for Complex Decision Making

Intelligent learning environments can be designed to support the development of learners' cognitive skills, strategies, and metacognitive processes as they work on complex decision-making and problem-solving tasks. However, the complexity of the tasks may impede the progress of novice learners. Providing adaptive feedback to learners who face difficulties requires learner modeling approaches that can identify learners' proficiencies and the difficulties they face in executing required skills, strategies, and metacognitive processes. This paper discusses a multilevel hierarchical learner modeling scheme that analyzes and captures learners' cognitive processes and problem-solving strategies along with their performance on assigned tasks in a game-based environment called UrbanSim that requires complex decision making for dealing with counterinsurgency scenarios. As the scenario evolves in a turn-by-turn fashion, UrbanSim evaluates the learners' moves using a number of performance measures. Our learner modeling scheme interprets the reported performance values by analyzing the learners' activities captured in log files to derive learners' proficiencies in associated cognitive skills and strategies, and updates the learner model. We discuss the details of the learner modeling algorithms in this paper, and then demonstrate the effectiveness of our approach by presenting results from a study we conducted at Vanderbilt University.