Evaluating large-scale training simulations

This article develops and describes a proposed training evaluation framework for large-scale training simulations (LSTS). These simulations may link together hundreds of participants at many different geographic locations and are more complex and costly than traditional training devices and simulators by an order of magnitude or more. The evaluation framework is based on a review and analysis of the literature pertaining to military training evaluation and ideas that have percolated up in the past dozen years from a small number of scientists engaged in LSTS evaluation. The analysis focuses on key evaluation questions: What is the purpose of the evaluation? Whose training is being evaluated? What is being evaluated? Where is the evaluation conducted? What evaluation methods are used? What are the evaluation criteria? What is the timing of evaluation events? The framework developed ties together evaluation events with prescribed objectives, timing, methods, and relevant evaluation criteria (dependent variables). It is intended to apply to any large-scale virtual, constructive, or advanced distributed simulation.