Pose-Motion Video Anomaly Detection via Memory-Augmented Reconstruction and Conditional Variational Prediction

Video anomaly detection (VAD) is a challenging computer vision problem. Due to the scarcity of anomalous events in training, the models learned by existing methods would mistakenly fit the ubiquitous non-causal or even spurious correlations, leading to failure in inference. In this paper, we propose a new two-phase Pose-Motion Video Anomaly Detection (PoMo) approach by jointly exploiting the informative features including the poses and optical flows that have rich causal correlations with abnormality. PoMo can effectively prevent the non-causal features from leaking in by either encoding only the essential information, i.e., the poses and optical flows, with our normalized autoencoder (phase one), or separately modeling the knowledge learned in phase one using our causal-conditioned autoencoder (phase two). The difference between normal and abnormal events can be amplified through these two phases. Thus the generalization ability can be reinforced. Extensive experimental results demonstrate the superiority of our approach over the existing methods and the improvements in AUC-ROC can be up to 1.5%.