StrongFusionMOT: A Multi-Object Tracking Method Based on LiDAR-Camera Fusion

This article proposes a multi-object tracking (MOT) method called StrongFusionMOT, which fuses the information of light detection and ranging (LiDAR) andcamera sensors. The major contributions of the proposed StrongFusionMOT are in three aspects. First, in the detection fusion stage, the depth information extracted by means of absolute difference (AD)-census is supplemented to 2-D detections to facilitate the fusion of 2-D and 3-D detections. This detection fusion pattern enhances fusion robustness and provides accurate fusion performance. Second, a new cost function design named shape-distance intersection over union (SDIoU) is proposed by taking into account not only the intersection between the two bounding boxes but also their shapes and relative distances. This cost function eliminates the shortcomings of the existing IoU designs and greatly enhances association precision. Third, a multiframe matching mechanism that involves tracks in the past n frames is proposed for reappeared tracks, which effectively suppresses cumulative errors resulting from consecutive frames of track predictions and greatly enhances association robustness. The effectiveness of the proposed StrongFusionMOT is evaluated by means of comparative experiments with the state-of-the-art MOT solutions using the KITTI dataset and the nuScenes dataset. Both quantitative and qualitative results demonstrate the superiorities of the proposed method in terms of various performance metrics.