ObjTest: Object-Level Mutation for Testing Object Detection Systems

With the tremendous advancement of deep learning techniques, object detection (OD) systems have achieved significant development. These systems, powered by deep neural networks, are now widely employed in diverse applications, including autonomous driving, intelligent video surveillance, and industrial inspection. Despite their impressive capabilities, OD systems, being complex software entities, can manifest erroneous behaviors that potentially lead to substantial losses. Moreover, the inherent complexity of detecting and localizing multiple objects in an image adds to the challenges of data annotation and system testing. To alleviate these challenges, in this paper, we propose ObjTest, an object-level mutation approach for testing OD systems. We generate large-scale test data by inserting, replacing, and removing target objects in the images while preserving their oracle information properly. We further propose an uncertainty evaluation metric for the prediction of test cases and adopt them to guide the test generation. Our comprehensive evaluation of ObjTest across three well-known OD datasets reveals that it effectively identifies numerous recognition failures. The results demonstrate that our object-level mutation approach yields more naturalistic alterations compared to traditional image-level transformations. Furthermore, the tests derived from our uncertainty metric-driven guidance enhance error detection efficiency and offer substantial benefits for guiding the retraining of OD systems to boost their performance.