Real-time human action prediction using pose estimation with attention-based LSTM network

Human action prediction in a live-streaming videos is a popular task in computer vision and pattern recognition. This attempts to identify activities in an image or video performed by a human. Artificial intelligence(AI)-based technologies are now required for the security and human behaviour analysis. Intricate motion patterns are involved in these actions. For the visual representation of video frames, conventional action identification approaches mostly rely on pre-trained weights of various AI architectures. This paper proposes a deep neural network called Attention-based long short-term memory (LSTM) network for skeletal based activity prediction from a video. The proposed model has been evaluated on the ‘BerkeleyMHAD’ dataset having 11 action classes. Our experimental results are compared against the performance of the LSTM and Attention-based LSTM network for 6 action classes such as Jumping, Clapping, Stand-up, Sit-down, Waving one hand (Right) and Waving two hands. Also, the proposed method has been tested in a real-time environment unaffected by the pose, camera facing, and apparel. The proposed system has attained an accuracy of 95.94% on ‘BerkeleyMHAD’ dataset. Hence, the proposed method is useful in an intelligent vision computing system for automatically identifying human activity in unpremeditated behaviour.