OSDet: Towards Open-Set Object Detection

In computer vision, almost all object detectors solve close-set object detection, assuming only known objects appear in the test environment. However, open-set object detection remains challenging since unknown objects need to be separated from the background unsupervised. To this aim, most existing state-of-the-art open-set detectors rely on the class-agnostic properties of the Region Proposal Network (RPN) to generate pseudo-labels for unknown objects. However, we notice that RPN generates low-quality pseudo-labels that significantly affect the performance of open-set detectors. Therefore, we propose Focusing on Location for Unknowns (FLU), which consists of class-agnostic pretraining and class-specific training, to improve the quality of pseudo-labels. Class-agnostic pretraining locates objects without learning to classify, and class-specific training generates high-quality pseudo-labels for unknowns. Besides, we implement a simple Object-level Contrastive Learning (OCL). OCL is helpful for the network to learn more discriminative features of objects, further reducing the confusion about known and unknown objects. With FLU and OCL, we present a novel open-set detector called OSDet. Ablations reveal the merits of FLU and OCL. Moreover, extensive experiments show that OSDet can significantly improve the performance of open-set detectors. e.g., OSDet reduces the Wilderness Impact by 20%similar to 25% on seven open-set object detection benchmarks.