Improving CNN-based semantic segmentation on structurally similar data using contrastive graph convolutional networks

Structurally similar data exist in most practical semantic segmentation applications. For example, objects can appear identical or positionally similar in many images, such as video frames. Objects with structural similarity in data samples can confuse deep neural networks (DNNs) in semantic segmentation applications. These challenges often lead to lower pixel classification accuracy of natural object segmentation. This study proposes a novel approach (S2-GCN) that enhances CNN -based semantic segmentation for structurally similar data using a contrastive graph convolutional network (GCN). By selecting specific label pairs and developing a customized GCN branch parallel to an encoder -decoder backbone, our method significantly improves accuracy, IoU, and F1score, by up to 8 %, as demonstrated through an extensive evaluation of five datasets. Our findings show that the proposed method effectively addresses the structural similarity problem of CNN -based semantic segmentation and can be applied to a wide range of practical applications.