Multi-guided feature refinement for point cloud semantic segmentation with weakly supervision

Point cloud segmentation is a widely concerned task for 3D part/scene parsing and many learning-based methods are proposed to improve the performance significantly. However, the performance is limited by the quality and quantity of labeled data. Therefore, we propose a multi-guided feature refinement (MGFR) to capture more effective representation with fewer annotations. Specifically, MGFR is a point-wise method based on a hybrid neighbor system and consists of feature aggregation and weight refinement. Feature aggregation is implemented in an attention-based manner guided by explicit information (structure geometry prior and RGB prior), neighbor information and prototype information. Weight refinement is a probabilistic method which is guided by the effective components of prototype extracted from neighbor members. The refined point feature of MGFR is provided with more local smoothness and global consistency, which can improve the performance on different instances of the same class and reduce the counterintuitive error around classification boundary or isolated outliers. Furthermore, we also use a neighbor-based contrastive loss, a prototype-based loss with regularization and a neighbor-based multiple instance loss to achieve local optimization and regularize the distribution of point embedding. Experimentally, we evaluate MGFR on ShapeNet Part dataset, Stanford 2D-3D (S3DIS) dataset and ScanNet, which shows the effectiveness in weakly-supervised task.