End-to-end multitask Siamese network with residual hierarchical attention for real-time object tracking

Object tracking with deep networks has recently achieved substantial improvement in terms of tracking performance. In this paper, we propose a multitask Siamese neural network that uses a residual hierarchical attention mechanism to achieve high-performance object tracking. This network is trained offline in an end-to-end manner, and it is capable of real-time tracking. To produce more efficient and generative attention-aware features, we propose residual hierarchical attention learning using residual skip connections in the attention module to receive hierarchical attention. Moreover, we formulate a multitask correlation filter layer to exploit the missing link between context awareness and regression target adaptation, and we insert this differentiable layer into a neural network to improve the discriminatory capability of the network. The results of experimental analyses conducted on the OTB, VOT and TColor-128 datasets, which contain various tracking scenarios, demonstrate the efficiency of our proposed real-time object-tracking network.