Detection of local and clustered outliers based on the density-distance decision graph

被引:15
|
作者
Li, Kangsheng [1 ]
Gao, Xin [1 ]
Jia, Xin [1 ]
Xue, Bing [1 ]
Fu, Shiyuan [1 ]
Liu, Zhiyu [1 ]
Huang, Xu [1 ]
Huang, Zijian [1 ]
机构
[1] Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing, Peoples R China
关键词
Outlier detection; Anomaly detection; Local reachable density; Kernel density estimation; Density lifting distance; Density-distance decision graph;
D O I
10.1016/j.engappai.2022.104719
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Outlier detection tasks refer to identifying the objects that have different characteristics from the normal observations. Most existing approaches detect outliers from the global perspective, which can effectively detect global outliers and most clustered outliers but cannot detect local outliers when the normal samples form clusters with different densities. The methods based on local outlier factors can effectively detect local outliers, but when the number of outliers increases, the more occurrences of clustered outliers will lead to the degeneration of the detection performance. We proposed an outlier detection method based on density-distance decision graph to detect local, global and clustered outliers simultaneously. Firstly, kernel density estimation and local reachable distance are combined to calculate the local density. The density ratio of the neighbors of an instance to itself is calculated as the degree of local outliers. Then, we propose a metric named density lifting distance as the degree of global outliers, which is calculated by the distance between k nearest neighbors with higher density of the instance and itself. The density ratio and density lift distance are combined to draw the density-distance decision graph, and the product of two metrics is calculated as the final outlier score. Comprehensive experiments were conducted on 8 synthetic datasets and 16 real-world datasets compared with 12 state-of-the-art methods. The results show that the proposed method works well when the samples form clusters with different densities as well as the percentage of outliers varies, and outperforms the state-of-the-art methods tested in terms of AUC.
引用
收藏
页数:15
相关论文
共 50 条
  • [31] Graph Anomaly Detection Based on Steiner Connectivity and Density
    Cadena, Jose
    Chen, Feng
    Vullikanti, Anil
    [J]. PROCEEDINGS OF THE IEEE, 2018, 106 (05) : 829 - 845
  • [32] A local density optimization method based on a graph convolutional network
    Wang, Hao
    Dong, Li-yan
    Fan, Tie-hu
    Sun, Ming-hui
    [J]. FRONTIERS OF INFORMATION TECHNOLOGY & ELECTRONIC ENGINEERING, 2020, 21 (12) : 1795 - 1803
  • [33] A local density optimization method based on a graph convolutional network
    Hao Wang
    Li-yan Dong
    Tie-hu Fan
    Ming-hui Sun
    [J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21 : 1795 - 1803
  • [34] An Efficient Distance and Density Based Outlier Detection Approach
    Zhong, Xunbiao
    Huang, Xiaoxia
    [J]. MECHANICAL ENGINEERING AND GREEN MANUFACTURING II, PTS 1 AND 2, 2012, 155-156 : 342 - 347
  • [35] A new local density and relative distance based spectrum clustering
    Mingzhe Liu
    Mingfu He
    Ruili Wang
    Shaoda Li
    [J]. Knowledge and Information Systems, 2019, 61 : 965 - 985
  • [36] A new local density and relative distance based spectrum clustering
    Liu, Mingzhe
    He, Mingfu
    Wang, Ruili
    Li, Shaoda
    [J]. KNOWLEDGE AND INFORMATION SYSTEMS, 2019, 61 (02) : 965 - 985
  • [37] Local community detection algorithm based on local modularity density
    Kun Guo
    Xintong Huang
    Ling Wu
    Yuzhong Chen
    [J]. Applied Intelligence, 2022, 52 : 1238 - 1253
  • [38] Local community detection algorithm based on local modularity density
    Guo, Kun
    Huang, Xintong
    Wu, Ling
    Chen, Yuzhong
    [J]. APPLIED INTELLIGENCE, 2022, 52 (02) : 1238 - 1253
  • [39] Outlier detection based on local minima density
    Liu, Jia
    Wang, Guoyin
    [J]. 2016 IEEE INFORMATION TECHNOLOGY, NETWORKING, ELECTRONIC AND AUTOMATION CONTROL CONFERENCE (ITNEC), 2016, : 718 - 723
  • [40] Detection of Similar Community in Large Network Based on Graph Edit Distance
    Narayan, Abhay
    Kumar, G. Santhosh
    [J]. PROCEEDINGS OF THE 2016 INTERNATIONAL CONFERENCE ON DATA SCIENCE & ENGINEERING (ICDSE), 2016, : 168 - 172