Robust online visual tracking via stable and adaptive memories

Visual tracking is a popular research topic in computer vision. In this paper, we propose a novel tracking framework which leverages stable and adaptive memories of target appearance for robust tracking where the target undergoes significant appearance change as well as background clutter. First, we define a stable-adaptive memory network which exploits the embedding of the target patch in the first video frame, named as "reliable memory", as well as the embeddings of patches collected online that are referred as "adaptive memories". Through the fusion of these two types of memories, a good balance between stability and plasticity can be made. During tracking, the network searches the candidate which has the highest similarity with the memorized patterns as the tracking result. Second, we train an online detector to re-detect the target in case of tracking failure and update the memory network. By virtue of the proposed mechanism, our tracker can handle the drift problem well and is able to track the object in challenging situations robustly. Experimental results on challenging benchmark video sequences show that the proposed tracking framework achieves state-of-the-art tracking performance with high accuracy and robustness.