A Spatio-Temporal Convolutional Neural Network for Skeletal Action Recognition

Human action recognition based on 3D skeleton data is a rapidly growing research area in computer vision. Convolutional Neural Networks (CNNs) have been proved to be the most effective representation learning in many vision tasks, but there is little work of CNNs for skeletal action recognition due to the variable-length of time sequences and lack of big skeleton datasets. In this paper, we propose a Spatio-Temporal CNN for skeleton based action recognition. A CNN architecture with two convolutional layers is used, in which the first layer is used to capture the spatial patterns and second layer for spatio-temporal patterns. Some techniques including data augmentation and segment pooling strategy are employed for long sequences. Experimental results on MSR Action3D, MSR DailyActivity3D and UT-Kinect show that our approach achieves comparable results with those of the state-of-the-art models.