Frame-level refinement networks for skeleton-based gait recognition

Gait is considered as a promising biometric feature due to its support for long-distance and non-contact recognition. Most existing skeleton-based gait recognition methods deliver the same topology of skeleton graph for each frame and treat them equally in the process of temporal feature extraction and fusion, which undoubtedly limits the expressive capability of the model. In this paper, we propose a frame-level refinement network to adaptively learn specific topology in different frames and capture long-range dependencies between frames through transformer self-attention. Specifically, we design a frame-level topology refinement graph convolution (FTR-GC) to dynamically model different correlations between joints for each frame. In addition, we introduce transformer self-attention at the frame level, which can learn inter-frame long-range relations between the same joint. Finally, an attention-based frame-level feature aggregation module (FFAM) is presented to produce discriminative global features of input gait sequences. Experiments on the popular public dataset CASIA-B and OUMVLP-Pose show that our method notably surpasses state-of-the-art model-based methods, verifying the effectiveness of the proposed modules.