Real-time urban street view semantic segmentation based on cross-layer aggregation network

With the rapid development of autonomous driving technology,precise and efficient scene un⁃ derstanding has become increasingly important. Urban street scene semantic segmentation aims to accu⁃ rately identify and segment elements such as pedestrians,obstacles,roads,and signs,providing necessary road information for autonomous driving technology. However,current semantic segmentation algorithms still face challenges in urban street scene segmentation,mainly manifested in issues such as insufficient dis⁃ crimination between different categories of pixels,inaccurate understanding of complex scene structures, and inaccurate segmentation of small-scale objects or large-scale structures. To address these issues,this paper proposed a real-time urban street scene semantic segmentation algorithm based on a cross-layer ag⁃ gregation network. Firstly,a pyramid pooling module combined with cross-layer aggregation was de⁃ signed at the end of the encoder to efficiently extract multi-scale context information. Secondly,a cross-layer aggregation module was designed between the encoder and decoder,which enhances the representa⁃ tion ability of information by introducing a channel attention mechanism and gradually aggregates the fea⁃ tures of the encoder stage to fully achieve feature reuse. Finally,a multi-scale fusion module was designed in the decoder stage,which aggregates global and local information in the channel dimension to promote the fusion of deep and shallow features. The proposed algorithm was validated on two common urban street scene datasets. On an RTX 3090 graphics card(TensorRT speed measurement environment),the algorithm achieves 73. 0% mIoU accuracy on the Cityscapes test set with real-time performance of 294 FPS,and 75. 8% mIoU accuracy on higher resolution images with real-time performance of 164 FPS;on the CamVid dataset,it achieves 74. 8% mIoU accuracy with real-time performance of 239 FPS. Experi⁃ mental results show that the proposed algorithm effectively balances accuracy and real-time performance, significantly improving semantic segmentation performance compared to other algorithms,and bringing new breakthroughs to the field of real-time urban street scene semantic segmentation. © 2024 Chinese Academy of Sciences. All rights reserved.