Generating actionable information for classifier assessment

DoD systems are becoming increasingly automated and classifiers are essential to their operation. Classifier assessment, then, is also essential, not only during tool selection, but also for maintaining deployed systems. Classifier evaluation is, however, not trivial: there are numerous classifier evaluation measures used. How does one select a measure, and once selected, what do the values mean? This article addresses what constitutes actionable information regarding a classifier's effect on mission capability, maps some commonly seen measures to these needs, and recommends a means of maximizing the actionable information generated in classifier evaluations.