Gait Recognition Based on Constrained Mutual Subspace Method with CNN Features

Gait recognition has various applications such as surveillance and criminal investigation since it can work even at a distance from a camera without the cooperation of subjects. In the conventional methods, especially when using silhouette images, a typical feature is Gait Energy Image (GEI), which is an average silhouette image of a given sequence. Although the GEI feature can represent the sequence compactly, it cannot capture a more detailed structure of the sequence. To address this issue and increase the robustness to the speed variations of walking, gait recognition based on Mutual Subspace Method (MSM) was proposed, where each image sequence is compactly represented by a subspace. In this paper, we enhance further the MSM based method by introducing two functions: 1) to add a feature extraction by projecting onto a generalized difference subspace, 2) to use Convolutional Neural Network (CNN) features, which are obtained from a fully connected layer of a learned CNN, as an input. The extended MSM with the projection is called Constrained MSM, which has been well known as a useful method for image set based recognition. The proposed method achieved the accuracy of 97.7% on an experiment with 1000 subjects from the OU-ISIR Large population datast.