Unsupervised Outlier detection algorithm based on k-NN and fuzzy logic

Given a set of observations an outlier is a measurement that differs significantly from other observations. In a real application, what is sought is to eliminate them since their processing implies statistical errors. Although there are several works that have addressed the outlier detection challenge, in recent works, efforts have been focused to unsupervised scenario because it does not require any a priori knowledge of data distributions and is more attached to reality. Unfortunately, unsupervised approaches have limitations under complex datasets. In order to solve this problem, we propose the use of K-NN rule and fuzzy logic for outlier detection. First, the proposed algorithm is evaluated by using synthetic data; after, the Harvard Unsupervised Anomaly Detection Benchmark Dataset, which consists of several complex data structures based in real-world applications, is used. In comparison with the current works, our algorithm outperforms most previous works for the Harvard Breast cancer dataset dataset (ROC score equal to 0.9980) while for the Harvard Pen Global dataset our algorithm achieves relatively higher accuracy (more accurate than some previous works) and similar results than most accurate algorithms in the current literature.