Evaluating Comprehension-Based User Models: Predicting Individual User Planning and Action

Described is a program of research that uses rigorous methods to evaluate models of user cognition and action based on the construction-integration architecture of comprehension (Doane and Sohn, 2000; Kintsch, 1988; 1998). The models interrelate user environmental information, background knowledge, and current goals, and then spread activation throughout the interrelated information to simulate UNIX user command productions, aviation pilot eye fixations and control movements during flight, and army personnel intelligence planning. Models of individuals in the complex interactive environments are tested for descriptive as well as predictive validity. Comparisons of model and human empirical data have resulted in a high degree of agreement, validating the ability of the comprehension-based architecture to support models that can predict user performance. Evaluation methods are detailed and the importance of evaluative rigor is discussed.