Geographic knowledge graph-guided remote sensing image semantic segmentation

Although the Deep Semantic Segmentation Network (DSSN) has notably enhanced remote-sensing image semantic segmentation, it still falls short of human experts' visual interpretation. Unlike DSSN's data-driven, pixel-level optimization, human experts rely on visual features, semantic insight, and prior knowledge for remote-sensing image interpretation. DSSN's pixel-level approach is constrained by spatial scale, lacking comprehensive target inference and struggling to bridge structured data and unstructured knowledge. In response to the two issues above, this paper proposes a geographic knowledge graph-guided deep semantic segmentation network for remote-sensing imagery. We use the ground-object semantic information and geoscience prior knowledge extracted from the geographic knowledge graph to construct loss constraints, thereby autonomously guiding the training process of DSSN. The essence of our approach lies in the intricately crafted design of loss constraints. These loss constraints include the entity-level connectivity constraint and the inter-entity symbiosis constraint. The former calculates the loss in the unit of connected domain entities instead of pixels to achieve overall constraints on the entity. The latter embeds the spatial symbiosis knowledge quantified by the symbiosis conditional probability into the data-driven DSSN to constrain the spatial distribution of segmented entities. The entity-level connectivity constraint guides DSSN to autonomously learn entity-level feature representations during training. Accordingly, the segmentation results become more holistic and suppresses blurry boundaries and random noise. The inter-entity symbiosis constraint adjusts the spatial distribution of entities according to the spatial semantic information and the prior geoscience knowledge. This adjustment realizes the automatic optimization of the spatial distribution of segmented entities. Extensive experiments show that under the guidance of the entity-level connectivity constraint and the inter-entity symbiosis constraint, DSSN can complete the learning of entity-level features. It can also automatically optimize the spatial distribution of ground objects based on spatial symbiosis knowledge, thereby effectively improving the performance of remote-sensing image semantic segmentation. Our novel geographic knowledge graph-guided approach to deep semantic segmentation in remote-sensing imagery has successfully addressed the challenges posed by DSSN's pixel-level optimization. By incorporating entity-level connectivity and inter-entity symbiosis constraints, we have enabled DSSN to autonomously learn comprehensive feature representations and optimize spatial distribution. The resulting improvements in semantic segmentation performance showcase the potential of merging domain-specific knowledge with data-driven techniques, bridging the gap between automated methods and human interpretation in remote-sensing image analysis. © 2024 Science Press. All rights reserved.