Multi-Stage Residual Fusion Network for LIDAR-Camera Road Detection

Only a few existing works exploit multiple modalities of data for road detection task in the context of autonomous driving. In this work, a deep learning based approach is developed to fuse LIDAR point cloud and camera image features over a bird's eye view representation. A two-stream fully-convolutional network is designed as encoder to extract general features of two types of data. Instead of limiting the fusion processing at a single stage or to a predefined extent, we propose a multi-stage residual fusion strategy to merge the feature maps in a residual learning fashion, and integrate the information at different network depth. Experiments conduct on KITTI road benchmark show that our proposed method has a significant improvement over single modality methods and other fusion approaches. And it is also among the top performing algorithms.