Centralized tracking and bidirectional long short-term memory for abnormal behaviour recognition

The contribution of this paper is an evaluation of the chain of calculations for a behaviour recognition method. That is, from person detection, centralized person tracking to a bi-directional long short-term memory (BiLSTM). The centralized person tracking fuses detections from distributed and multimodal sensors. The BiLSTM learns long-time dependencies in the tracking data sequences. We use experimental sensor data from visual and thermal infrared sensors. The sensor data describe five scenarios with people performing normal and abnormal behaviours. The results indicate that the mean recognition accuracy is rather high. However, with position as the only input data, the robustness of the method is rather low. The robustness increases by adding velocity to the dataset. Velocity adds important information, even though velocity appears very messy when visualized in diagrams. Furthermore, the BiLSTM is compared with the unidirectional long short-term memory (LSTM) and the gated recurrent unit (GRU).