The impact of expert-system-based training on calibration of decision confidence in emergency management

In many emergency incidents, human operators need to derive countermeasures based an contingency rules under time pressure. Since people tend to be overconfident regarding their performance levels, it ir necessary that the operators be well trained to calibrate proper decision confidence in the safety-related domain. This paper examines the effectiveness of using expert systems to train for the desired calibration. Emergency management of chemical spills was selected to exemplify the rule-based decision task. An expert system in the domain was developed to serve as the training tool. A total of 40 student subjects participated in an experiment in which they were asked to resolve spill scenarios under the manipulation of training and deadline conditions. The experiment results indicate that people tend to overestimate their performance capabilities when reasoning with a rule-based knowledge set, especially with the presence of time constraints. The results also show that the manifestation of overconfidence can be reduced for individuals who undergo the expert-system calibration training. The implications of the findings are examined in this paper. (C) 1997 Elsevier Science Ltd.