Semantic Segmentation Model for Remote Sensing Images Combining Super Resolution and Domain Adaption

The semantic segmentation method based on convolutional neural network rely on supervised learning with ground truth, but it cannot be well generalized to unlabeled datasets with different sources. Unsupervised domain adaptation can solve the problem of inconsistent feature distribution between unlabeled target domain data and labeled source domain data. This is due to that remote sensing images are often come from different sources, they are variable in their spatial resolution and are influenced by different imaging regions, imaging conditions and imaging times. Even images from the same region may have large differences in spectral features. The generalization of the semantic segmentation model relies on the reduction of these inter-domain differences mentioned above. Therefore, unsupervised domain adaptation methods for remote sensing image should not only reduce the differences in features between domains, but also address the problem of different spatial resolutions. This paper designs a new end-to-end semantic segmentation deep network combined with image super resolution-Semantic Segmentation Model Combining Super Resolution and Domain Adaption, which can reduce the spatial resolution difference and feature distribution difference between the low spatial resolution source domain and high spatial resolution target domain remote sensing images, and accomplish the super-resolution task for the source domain and the domain adaptation semantic segmentation task for the target domain. The SSM-SRDA model consists of three parts one is Semantic Segmentation Network with Super Resolution (SSNSR), the second is Pixel-level Domain Discriminator (PDD), and the third is Output-space Domain Discriminator (ODD). SSNSR consists of a semantic segmentation network and a super-resolution network, which share the same feature extraction network. The super-resolution network generates a high-resolution synthetic image with target image style from a low-resolution source domain image, which can eliminate spatial resolution differences and style differences to help the feature extraction module learn the same features between the source and target domains. The Feature Affinity-Loss module enhances the learning of the semantic segmentation deep network by the feature maps with more detailed structural information obtained by super-resolution. The pixel-level domain discriminator is used to reduce the pixel-level feature differences between the high-resolution target domain and the synthetic image of the source domain. High-resolution source domain synthetic images with target domain style are generated by generative adversarial learning with the super-resolution network and participate in the training of the model as additional training data. The output-space domain discriminator reduces the feature differences between source and target domain images in the output space of the semantic segmentation network. Through the adversarial learning of the semantic segmentation network and the two discriminators, SSM-SRDA aligns the feature distribution of the source domain and the target domain at the input and output stages of the segmentation network, and can be applied to the target domain datas of more different sources through domain adaptation. It is a practical and more popular model. Experiments show that SSM-SRDA is superior to the existing domain adaptive semantic segmentation methods on four sets of remote sensing image data sets, and the intersection ratio is improved by 0.7%, 1.7%, 2.2% and 3.3%, respectively. © 2022, Science Press. All right reserved.