Human Fall-down Event Detection Based on 2D Skeletons and Deep Learning Approach

The goal of this research is to apply the state-of-the-art deep learning approach to human fall down event detection based on 2D skeletons extracted from RGB sequence. In this paper, we adopt convolutional neural network (CNN) to extract humans' 2D skeletons for each input frame and then employ recurrent neural network (RNN) with Long Short Term Memory (LSTM) state cells to process temporal skeleton series to make best use of not only spatial features but also temporal information to classify each short-term action to five categories, i.e., standing, walking, falling, lying, and rising. After simple rule processing, the consecutive RNN outputs can be used to detect human's long-term actions (falling down event) and determine whether to issue an alarm or not. The accuracy of classification into 5 sub-actions is capable of achieving 90%. Our contributions lie on two aspects: (1) improving the performance on short-term human action recognition based on the combination of CNN and RNN/LSTM, (2) excluding the fall-down events that actually need no help and achieving a lower false alarm rate.