A Cognitive Memory-Augmented Network for Visual Anomaly Detection

With the rapid development of automated visual analysis, visual analysis systems have become a popular research topic in the field of computer vision and automated analysis. Visual analysis systems can assist humans to detect anomalous events (e.g., fighting, walking alone on the grass, etc). In general, the existing methods for visual anomaly detection are usually based on an autoencoder architecture, i.e., reconstructing the current frame or predicting the future frame. Then, the reconstruction error is adopted as the evaluation metric to identify whether an input is abnormal or not. The flaws of the existing methods are that abnormal samples can also be reconstructed well. In this paper, inspired by the human memory ability, we propose a novel deep neural network (DNN) based model termed cognitive memory-augmented network (CMAN) for the visual anomaly detection problem. The proposed CMAN model assumes that the visual analysis system imitates humans to remember normal samples and then distinguishes abnormal events from the collected videos. Specifically, in the proposed CMAN model, we introduce a memory module that is able to simulate the memory capacity of humans and a density estimation network that can learn the data distribution. The reconstruction errors and the novelty scores are used to distinguish abnormal events from videos. In addition, we develop a two-step scheme to train the proposed model so that the proposed memory module and the density estimation network can cooperate to improve performance. Comprehensive experiments evaluated on various popular benchmarks show the superiority and effectiveness of the proposed CMAN model for visual anomaly detection comparing with the state-of-the-arts methods. The implementation code of our CMAN method can be accessed at https://github.com/CMANcode/CMAN_pytorch.