Perturbation consistency and mutual information regularization for semi-supervised semantic segmentation

Recent semi-supervised learning has attracted much attention by leveraging the hidden structures learned from unlabeled data to reduce the number of required labels in the field of human-centric understanding. Most semi-supervised methods have been proposed to improve the performance of image classification, and their ideas cannot be directly applied to the task of semantic segmentation. In this paper, we propose a semi-supervised model for sematic segmentation named semi-supervised consistency segmentation (SCSeg). The performance gain benefits from two techniques-perturbation consistency and mutual information regularization. Perturbation consistency enforces the output consistency between the uncorrupted and perturbed features. Mutual information regularization adopts a mutual information loss to ensure the spatial consistency of adjacent patches on unlabeled data. The experimental results on Pascal VOC 2012 and Cityscapes datasets widely used in visual understanding tasks demonstrate that the proposed model outperforms the current semi-supervised segmentation methods under varying amounts of labeled data. The proposed model alleviates the pressure of annotation in human-centric practical multimedia applications towards semantic segmentation.