A point-based deep learning network for semantic segmentation of MLS point clouds

Semantic segmentation of point cloud is critical to 3D scene understanding and also a challenging problem in point cloud processing. Although an increasing number of deep learning based methods are proposed in recent years for semantic segmentation of point clouds, few deep learning networks can be directly used for large-scale outdoor point cloud segmentation which is essential for urban scene understanding. Given both the challenges of outdoor large-scale scenes and the properties of the 3D point clouds, this paper proposes an end-to-end network for semantic segmentation of urban scenes. Three key components are encompassed in the proposed point clouds deep learning network: (1) an efficient and effective sampling strategy for point cloud spatial downsampling; (2) a point-based feature abstraction module for effectively encoding the local features through spatial aggregating; (3) a loss function to address the imbalance of different categories, resulting in the overall performance improvement. To validate the proposed point clouds deep learning network, two datasets were used to check the effectiveness, showing the state-of-the-art performance in most of the testing data, which achieves mean IoU of 70.8% and 73.9% in Toronto-3D and Shanghai MLS dataset, respectively.