Active localization learning for weakly supervised instance segmentation

Instance segmentation models trained with weak annotations offer a more cost-effective alternative to fully supervised models. However, there remains a nearly 30% performance gap, which significantly hinders their applicability in real-world scenarios. To narrow this gap, we propose a novel adversarial modulation module designed to extract instance localization distributions. This module addresses the unclear stopping conditions for continuously contracting sub-discriminative noise activation during localization. When the contraction and expansion mechanisms reach a balanced state, the localization process is automatically terminated. Furthermore, to break mere 0.3% performance gain from the complementary fusion of different localization branches, we revisit the issue of localization mismatch and propose a unified "localization mismatch" metric (LM). LM, an active learning strategy, aims to automatically collect well-documented failure patterns of a pre- trained weakly supervised instance segmentation (WSIS) model. These value error patterns are specifically selected for annotation and fine-tuning to enhance WSIS performance. Performance tests on the Pascal VOC2012 and COCO datasets show that by actively labeling only 5% of valuable images in Pascal VOC2012, our approach achieves 95% of the performance of a fully supervised Panoptic-DeepLab. In the COCO dataset, by actively labeling an average of 10 fully annotated images per category, which accounts for 3% of the training set, LM bridges more than 75% of the performance gap in the mAP score. Our code can be accessed publicly at https://github.com/Elaineok/ALL.