Open-Set Recognition for Automatic Target Recognition: Practical Considerations for Obtaining Out of Distribution Examples

In most image classification applications, the task is assumed to be "closed-set" in which the only classifications the model expects to make are of examples that it was originally trained on. However, the real world presents a much more complex "open-set" in which a given model may encounter examples it was not trained to classify. Open-Set Recognition is the practice of enabling classifiers to recognize when they have encountered a given example that they were not previously trained to classify. Typically, these Open-Set Recognition techniques can be grouped into two categories: those that require a feature space, and those that learn a feature space. However, finding a suitable feature space is difficult and so it is often necessary that one is learned. To accomplish this, one can leverage "Out of Distribution" examples, or examples that exist outside of the training data. This effort explores the various methods of obtaining Out of Distribution examples and how they compare. Additionally, based on our findings, we make practical recommendations for obtaining Out of Distribution examples to enable Open-Set Recognition techniques for overhead imagery and Synthetic Aperture Radar (SAR) applications.