Methods for Non-Intrusive Out-Of-Distribution Images Detection

Selecting representative data is a key factor in improving the performance of machine learning algorithms. In this paper we focus on out-of-distribution (OoD) methods evaluation, which can be integrated into ML project lifecycle in a non-intrusive way, without changing a model architecture. Considered methods are applicable to image classification datasets analysis. In addition to commonly used AUROC metric, we evaluate the number of out-of-distribution samples misclassified with high confidence. Case studies were conducted on benchmark and production datasets. As a result, we provide practical guidance for data evaluation and recommendations on which method to use to detect different types of OoD images.