An experiment in agent teaching by subject matter experts

This paper presents a successful knowledge-acquisition experiment in which subject matter experts who did not have any prior knowledge-engineering experience succeeded in teaching the Disciple-COA agent how to critique courses of action, a challenge problem addressed by the DARPA's High-Performance Knowledge Bases program. We first present the COA challenge problem and the architecture of Disciple-COA, a learning agent shell from the Disciple family. Then we present the knowledge acquisition experiment, detailing both the expert-Disciple interactions, and the automatic knowledge base development processes that take place as a result of these interactions. The results of this experiment provide strong evidence that the Disciple approach is a viable solution to the knowledge acquisition bottleneck. (C) 2000 Academic Press.