Dual-Stream Feature Fusion Network for Detection and ReID in Multi-object Tracking

Multi-object Tracking (MOT) focuses on associating detection boxes with previous results from the same detector using motion and appearance features. Recently, the Joint detection and embedding (JDE) paradigm is showing its efficiency to explore the potentially shared information between the tasks of detection and re-identification (ReID). However, formulating the tasks of detection and ReID into a shared feature fusion network may bring inevitable competition during training. To tackle this problem, we propose a novel dual-stream feature fusion network (DSFFN) to alleviate the competition between the two tasks and obtain better task-dependent feature expression. It forms the detection and ReID feature fusion tasks into a parallel network, and they only share the low-level feature information. Further, to solve the inconsistency across different scales of ReID embeddings and improve the awareness of important information, we propose a multi-scale cross-connected attention network (MSCCAN) for ReID feature fusion. At last, we modify the prediction head to be decoupled design to tackle the conflicts between multi-task. Our method obtains 78.8% MOTA and 74.3% IDF1 on MOT16 test sets, which outperforms the previous works and achieves state-of-the-art performance.