Human Activity Recognition Using Combined Deep Architectures

Human activity recognition has been an active area of research in computer vision and artificial intelligence since the last two decades. This research aims to combine color information from RGBD with motion information from skeletal joints in order to capture subtle motions in the video which in used to tackle temporal features extraction. The researchers show this by using combined deep architecture: Convolutional Neural Network (CNN) and Stacked Auto-encoders (SAE). In this model CNN is used to learn motion representations from Motion History Images (MHIs) of sampled RGB image frames. Whereas SAE is used to learn discriminative movements of human skeletal joints by taking the distances of joints from mean joint at each sampled frame. The proposed model is able to learn low level abstractions of joint motion sequences as well as how motion changes with location of image in each MHI frame. Both deep architectures were trained separately by averaging the Softmax class posteriors across the sampled frames to obtain score of the video clip. Normalize class scores of each of the networks in [0,1] range and perform late fusion by taking weighted mean of class scores based on relative performances of the two networks. The proposed model is evaluated on standard action recognition benchmarks of MSR Daily Activity3D and MSR Action3D datasets, where the proposed architecture has improved recognition accuracy.