A method for behaviour recognition based on long short-term memory (LSTM) and splines

The paper describes a method for human behaviour recognition in surveillance applications. The method is based on the long short-term memory (LSTM), which is a form of recurrent neural network (RNN). LSTM has a memory function that can learn long-term dependencies by remembering short-term information for a long time. LSTM is therefore suitable for the events that are of interest in this paper. Except for the LSTM, the method also includes a simulation model for producing training and validation datasets. The simulation model describes a real environment with streets, buildings and squares and the motion patterns are represented with mathematical spline curves. The amount of training data are expanded further by creating different small variations of the spline curves. Output from the simulation model are matrices consisting of position, velocity and acceleration for a selected simulation time and sampling frequency. Before the data are used as input data to the LSTM a scaling is done so that the data patterns representing position, velocity and acceleration can contribute fully with useful information to the training process. The paper presents and discusses the method as well as results in the form of recognition accuracy of some simulated surveillance scenarios.