Graph Convolutional Networks-Hidden Conditional Random Field Model for Skeleton-Based Action Recognition

Recently, Graph Convolutional Network(GCN) methods for skeleton-based action recognition have achieved great success due to their ability to preserve structural information of the skeleton. However, these methods abandon the structural information in the classification stage by employing traditional fully-connected layers and softmax classifier, leading to sub -optimal performance. In this work, a novel Graph Convolutional Networks-Hidden conditional Random Field (GCN-HCRF) model is proposed to solve this problem. The proposed method combines GCN and HCRF to retain the human skeleton structure information during the classification stage. The proposed model is trained end -to -end by utilizing message passing from belief' propagation algorithm on the human structure graph. To further capture spatial and temporal information, we propose a multi -strewn framework which takes the relative coordinates of the joints and hone direction as two static feature streams, and the temporal displacements as the dynamic feature stream. Experimental results on two challenging benchmarks (NTU RGB+D, N-UCLA) show the superior performance of the proposed model over state-of-theart models.