Geometric Deep Learning on Skeleton Sequences for 2D/3D Action Recognition

Deep Learning models, albeit successful on data defined on Euclidean domains, are so far constrained in many fields requiring data which underlying structure is a non-Euclidean space, namely computer vision and imaging. The purpose of this paper is to build a geometry aware deep learning architecture for skeleton based action recognition. In this perspective, we propose a framework for non-Euclidean data classification based on 2D/3D skeleton sequences, specifically for Parkinson's disease classification and action recognition. As a baseline, we first design two Euclidean deep learning architectures without considering the Riemannian structure of the data. Then, we introduce new architectures that extend Convolutional Neural Networks (CNNs) and Recurrent Neural Networks(RNNs) to non-Euclidean data. Experimental results show that our method outperforms state-of-the-art performances for 2D abnormal behavior classification and 3D human action recognition.