Scale-fusion framework for improving video-based person re-identification performance

Video-based person re-identification (re-id), which aims to match people through videos captured by non-overlapping camera views, has attracted lots of research interest recently. In this paper, we first propose a novel hybrid 2D and 3D convolution-based recurrent neural network (HCRN) for video-based person re-id task. Specifically, the 3D convolutional module can explore the local short-term fast-varying motion information, while the recurrent layer can leverage the global long-term spatial–temporal information. Based on HCRN, we design a scale-fusion framework to make full use of features of different scales to further improve the performance of video-based person re-id. More concretely, the scale-fusion framework preserves a complete subnetwork similar to HCRN for each scale to extract features and exchanges information between all subnetworks at several stages of the framework. Besides, we propose a training method called species invasion to further improve the performance of HCRN and scale-fusion framework by utilizing a large amount of unlabeled data. Experimental results on the publicly available PRID 2011, iLIDS-VID and MARS multi-shot pedestrian re-id datasets demonstrate the effectiveness of the proposed HCRN, scale-fusion framework and species invasion training method.