A Real-time Anomalies Detection System based on Streaming Technology

With the wide deployment of flow monitoring in IP networks, flow data has been more and more applied on abnormal traffic detection. In practice, anomalies should be detected as fast as possible from giant quantity of flow data, while, at present, some classical anomalies detecting methods can not achieve this goal. In this paper, we propose and implement a distributed streaming computing system which aims to perform real-time anomalies detection by leveraging Apache Storm, a stream-computing platform. Based on this efficient system, we can uninterruptedly monitor the mutation of flow data and locate the source of anomalies or attacks in real-time by finding the specific abnormal IP addresses. A typical application example proved the capability and benefits of our system and we also have a detailed discussion in performance measurements and scalability.