A Frustum-based probabilistic framework for 3D object detection by fusion of LiDAR and camera data

This paper presents a real-time 3D object detector based on LiDAR based Simultaneous Localization and Mapping (LiDAR-SLAM). The 3D point clouds acquired by mobile LiDAR systems, within the environment of buildings, are usually highly sparse, irregularly distributed, and often contain occlusion and structural ambiguity. Existing 3D object detection methods based on Convolutional Neural Networks (CNNs) rely heavily on both the stability of the 3D features and a large amount of labelling. A key challenge is efficient detection of 3D objects in point clouds of large-scale building environments without pre-training the 3D CNN model. To project image-based object detection results and LiDAR-SLAM results onto a 3D probability map, we combine visual and range information into a frustum-based probabilistic framework. As such, we solve the sparse and noise problem in LiDAR-SLAM data, in which any point cloud descriptor can hardly be applied. The 3D object detection results, obtained using both backpack LiDAR dataset and the well-known KITTI Vision Benchmark Suite, show that our method outperforms the state-of-the-art methods for object localization and bounding box estimation.