A Shared Representation for Object Tracking and Classification using Siamese Networks

Recently, Siamese neural networks have been employed to build several high performance object trackers capable of operating in real time. To further improve the tracking performance, one can train one network on the tracking task and another network on the task of object classification. One can then use the feature representations of both networks to obtain a tracker which performs better than each network on its own. This approach, however, has the downside that two networks have to be evaluated instead of one, resulting in runtime degradation. We demonstrate that it is feasible to train one Siamese network on the tracking and the classifications tasks simultaneously. Specifically, we achieve a tracking performance similar to the performance of two networks trained on tracking and classification separately. Since our approach does not depend on two separate networks though, it allows one to improve the performance of a Siamese network tracker without any runtime penalty.