Comprehensive mining of information in Weakly Supervised Semantic Segmentation: Saliency semantics and edge semantics

In the studies of Weakly Supervised Semantic Segmentation (WSSS) with image-level labels, there is an issue of incomplete semantic information, which we summarize as insufficient saliency semantic mining and neglected edge semantics. We proposes a two-stage framework, Saliency Semantic Full Mining-Edge Semantic Mining (SSFM-ESM), which views WSSS from the perspective of comprehensive information mining. In the first stage, we rely on SSFM to address the insufficient saliency semantic mining. The network learns feature representations consistent with salient regions via the proposed pixel-level class-agnostic distance loss. Then, the full saliency semantic information is mined by explicitly receiving pixel-level feedback. The initial pseudo label with complete saliency semantic information can be obtained after the first stage. In the second stage, we focus on the mining of edge semantic information through the proposed edge semantic mining module. Specifically, we guide the initial pseudo-label avoid learning about false semantic information and obtain high-confidence edge semantics. The self-correction ability of the segmentation network is also fully utilized to obtain more edge semantic information. Extensive experiments are conducted on the PASCAL VOC 2012 and MS COCO 2014 datasets to verify the feasibility and superiority of this approach.