Distinguishing false from true alerts in Snort by data mining patterns of alerts

The Snort network intrusion detection system is well known for triggering large numbers of false alerts. In addition, it usually only warns of a potential attack without stating what kind of attack it might be. This paper presents a clustering approach for handling Snort alerts more effectively. Central to this approach is the representation of alerts using the Intrusion Detection Message Exchange Format, which is written in XML. All the alerts for each network session are assembled into a single XML document, thereby representing a pattern of alerts. A novel XML distance measure is proposed to obtain the distance between two such XML documents. A classical clustering algorithm, implemented based on this distance measure, is then applied to group the alert patterns into clusters. Our experiment with the MIT 1998 DARPA data sets demonstrates that the clustering algorithm can distinguish between normal sessions that give rise to false alerts and those sessions that contain real attacks, and in about half of the latter cases can effectively identify the name of the attack.