(e-mail:)

被引:0
|
作者
Xia, Weidai [1 ]
Zhou, Dongming [1 ]
Cao, Jinde [2 ,3 ]
Liu, Yanyu [1 ]
Hou, Ruichao [4 ]
机构
[1] Yunnan Univ, Sch Informat Sci & Engn, Kunming 650091, Peoples R China
[2] Southeast Univ, Sch Math, Nanjing 210096, Peoples R China
[3] Yonsei Univ, Yonsei Frontier Lab, Seoul 03722, South Korea
[4] Nanjing Univ, Sch State Key Lab Novel Software Technol, Nanjing 210023, Peoples R China
关键词
RGBT tracking; Cross-modality; Multi-level modality-shared fusion; TRACKING; NETWORK;
D O I
10.1016/j.neucom.2022.04.017
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
RGBT tracking is receiving more and more attention because of its huge tracking potential in an allweather environment. RGB and thermal source data contain different levels of information about the object. Utilizing the complementary advantage of different levels of information can effectively improve the tracking performance. Existing work focuses on the extraction and fusion of multi-modal features. Although these methods effectively deploy the fusion of information among multiple modalities, they ignore the potential value of multi-level shared clues in different modalities. In addition, these works cannot provide effective candidate boxes after tracking drift, resulting in limited tracker performance. In this paper, we propose a cross-modality interaction and re-identification network that performs multi-level modality-shared, modality-specific and object probability prediction learning. We designed two feature extraction sub-networks, namely, a multi-level modality-shared fusion network and modality complementary sub-network. Specifically, the two sub-networks extract and fuse multi-level modality shared information and modality specific information, respectively. To optimize tracking drift, object-aware branches that predict the object-centered state are designed. Our object-aware branching is simple, neat and efficient. Moreover, to achieve the visual tracking real-time requirement, we designed the object regression branch that does not require repeated region suggestion input. By extensive experiments and comparisons with state-of-the-art trackers on the RGBT tracking benchmark dataset, our tracker achieves leading performance and essentially real-time tracking speeds. Tracking drift caused by occlusion, fast motion and camera moving is significantly optimized. (c) 2022 Elsevier B.V. All rights reserved.
引用
收藏
页码:327 / 339
页数:13
相关论文
共 50 条