An entropy-based unsupervised anomaly detection pattern learning algorithm

Currently, most anomaly detection pattern learning algorithms require a set of purely normal data from which they train their model. If the data contain some intrusions buried within the training data, the algorithm may not detect these attacks because it will assume that they are normal. In reality, it is very hard to guarantee that there are no attack items in the collected training data. Focusing on this problem, in this paper, firstly a new anomaly detection measurement is proposed according to the probability characteristics of intrusion instances and normal instances. Secondly, on the basis of anomaly detection measure, we present a clustering-based unsupervised anomaly detection patterns learning algorithm, which can overcome the shortage above. Finally, some experiments are conducted to verify the proposed algorithm is valid.