A Quad Joint Relational Feature for 3D Skeletal Action Recognition with Circular CNNs

To deal with the limitations of human action recognition systems that apply deep neural networks (DNNs) to 3D skeletal feature maps, we propose an improved set of features that enable better pattern discrimination when using a spectrally enriched circular convolutional neural network (CCNN). These new features exploit the local relationships between joint movements based on 3D quadrilaterals constructed for all possible sets of four joints. Next, we compute the volumes of these time-varying quadrilaterals, by generating color-coded images, named spatio-temporal quad-joint relative volume feature maps (QjRVMs). To preserve the pixel frequency distribution while training a DNN, which is otherwise lost due to vanishing gradients and random dropouts, we propose a new architecture CCNNs. CCNNs use cyclic multi-resolution filters in a four-stream architecture, requiring only batch normalization and ReLU operations to identify multiple pixel pattern variations simultaneously. Applying the proposed CCNN to QjRVM images illustrates that combining multi-resolution features enhances the overall classification accuracy. Finally, we evaluate our proposed human action framework using our own 102-class, 5-subject action dataset, created using 3D motion capture technology, named KLHA3D-102. We also evaluate our framework using 3 publicly available datasets: CMU, HDM05, and NTU RGB-D.