Toward an Efficient Real-Time Anomaly Detection System for Cloud Datacenters

Anomaly detection in streaming data of cloud datacenter environments requires efficient real-time systems and algorithms. This paper proposes the Decreased Anomaly Score by Repeated Sequence (DASRS) algorithm, which normalizes time series values and counts each sequence to generate anomaly scores as a function of the number of times they appear. We also propose and implement the Sophia anomaly detection system. Sophia is a big data modular streaming processing system implemented in the Globo.com cloud datacenter. DASRS achieves the best-in-class score calculated by Numenta Anomaly Benchmark (NAB) framework. Besides, it is the fastest and uses the least memory among the state-of-the-art algorithms included in NAB. Results from a live application show that Sophia provides an accurate real-time anomaly detection service.