Policy-based reinforcement learning for time series anomaly detection

Time series anomaly detection has become a crucial and challenging task driven by the rapid increase of streaming data with the arrival of the Internet of Things. Existing methods are either domain-specific or require strong assumptions that cannot be met in realistic datasets. Reinforcement learning (RL), as an incremental self-learning approach, could avoid the two issues well. However, the current investigation is far from comprehensive. In this paper, we propose a generic policy-based RL framework to address the time series anomaly detection problem. The policy-based time series anomaly detector (PTAD) is progressively learned from the interactions with time-series data in the absence of constraints. Experimental results show that it outperforms the value-based temporal anomaly detector and other state-of-the-art detection methods whether training and test datasets come from the same source or not. Furthermore, the tradeoff between precision and recall is well respected by the PTAD, which is beneficial to fulfill various industrial requirements.