Evaluation of fusion-based ATR technology

Reliance on Automated Target Recognition (ATR) technology is essential to the future success of Intelligence, Surveillance, and Reconnaissance (ISR) missions. Although benefits may be realized through ATR processing of a single data source, fusion of information across multiple images and multiple sensors promises significant performance gains. A major challenge, as ATR fusion technologies mature, will be the establishment of sound methods for evaluating ATR performance in the context of data fusion. This paper explores the issues associated with evaluations of ATR algorithms that exploit data fusion. Three major areas of concern are examined, as we develop approaches for addressing the fusion-based evaluation problem: Characterization of the testing problem: The concept of operating conditions, which characterize the test problem, requires some generalization in the fusion setting. For example, conditions such as articulation or model variant, which are of concern for synthetic aperture radar (SAR) data, may be of minor importance for hyperspectral imaging (HSI) methods. Conversely, solar illumination conditions, which have no effect on the SAR signature, will be critical for spectral recognition. In addition, the fusion process may introduce critical for spectral based target new operating conditions, such as registration accuracy. Developing image truth and scoring rules: The introduction of multiple data sources raises questions about what constitutes successful target detection. Ground truth must be associated with multiple data sources to score performance. Performance metrics: New performance metrics, that go beyond simple detection, identification, and false alarm rates, are needed to characterize performance in the context of image fusion. In particular, algorithm developers would benefit from an understanding of the salient features from each data source and how these features interact to produce the observed system performance.