Improving Deep Multi-modal 3D Object Detection for Autonomous Driving

Object detection in real-world applications such as autonomous driving scenarios is a challenging issue since objects often occlude each other. 3D object detection has achieved high accuracy and efficiency, but detecting small object instances and occluded objects are the most challenging issues to deploy detectors in crowded scenes. Our main focus in this paper is deep multi-modal based object detector in an automated driving system with early fusion on 3D object detection utilizing both Light Detection and Ranging (LiDAR) and image data. We aim at obtaining highly accurate 3D localization and recognition of objects in the road scene and try to improve the performance. In this regard, our basic architecture follows an established two-stage architecture, Aggregate View Object Detection-Feature Pyramid Network (AVOD-FPN), one of the best among sensor fusion-based methods. AVOD- FPN has yielded promising results especially for detecting small instances. Moreover, another main challenging issue in autonomous driving is detecting the occluded objects. So we try to address this difficulty by integrating attention network into the multi-modal 3D object detector. Experiments are shown to produce state-of-the-art results on the KITTI 3D sensor fusion-based object detection benchmark.