Big data outlier detection model based on improved density peak algorithm

被引:10
|
作者
Shao, Mengliang [1 ,2 ]
Qi, Deyu [2 ]
Xue, Huili [3 ]
机构
[1] Guangzhou Univ, South China Inst Software Engn, Dept Comp Sci, Guangzhou, Peoples R China
[2] South China Univ Technol, Res Inst Comp Syst, Guangzhou, Guangdong, Peoples R China
[3] Guangzhou Nanyang Polytech Coll, Sch Informat Engn, Guangzhou, Peoples R China
基金
中国国家自然科学基金;
关键词
Outlier detection; big data; KNN algorithm; density clustering; CLUSTERING-ALGORITHM; MIXTURE MODEL; BEHAVIOR;
D O I
10.3233/JIFS-189456
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Outlier detection is an important branch of data mining. This paper proposes an advanced fast density peak outlier detection algorithm based on the characteristics of big data. The algorithm is an outlier detection method based on the improved density peak clustering algorithm. This paper improves the original algorithm. From the perspective of outlier detection, although it is a clustering idea, it avoids the clustering process, reduces the time complexity of the cluster-based outlier detection algorithm, and absorbs. The outlier detection based on neighbors is not sensitive to data dimensions and other advantages. In the power industry, outlier detection can be used in areas such as grid fault detection, equipment fault detection, and power abnormality detection. The simulation experiment of outlier detection based on the daily load curve of single and multiple transformers in a certain province shows that the improved algorithm can effectively detect outliers in the data.
引用
收藏
页码:6185 / 6194
页数:10
相关论文
共 50 条
  • [1] DP_DETECTION: An outlier detection algorithm based on density of big data
    Li, Xiaodi
    Deng, Ping
    Huang, Ming
    Li, Dingcheng
    Wang, Hongjun
    [J]. DATA SCIENCE AND KNOWLEDGE ENGINEERING FOR SENSING DECISION SUPPORT, 2018, 11 : 534 - 544
  • [2] A distributed density-based outlier detection algorithm on big data
    Mei, Lin
    Zhang, Fengli
    [J]. International Journal of Network Security, 2020, 22 (05): : 775 - 781
  • [3] An efficient algorithm for distributed density-based outlier detection on big data
    Bai, Mei
    Wang, Xite
    Xin, Junchang
    Wang, Guoren
    [J]. NEUROCOMPUTING, 2016, 181 : 19 - 28
  • [4] Outlier detection algorithm based on fast density peak clustering outlier factor
    Zhang, Zhongping
    Li, Sen
    Liu, Weixiong
    Liu, Shuxia
    [J]. Tongxin Xuebao/Journal on Communications, 2022, 43 (10): : 186 - 195
  • [5] Big Data Outlier Detection Algorithm Based on Grid
    Guo Wei-Wei
    Liu Feng
    [J]. 2018 11TH INTERNATIONAL CONFERENCE ON INTELLIGENT COMPUTATION TECHNOLOGY AND AUTOMATION (ICICTA 2018), 2018, : 274 - 277
  • [6] A New Outlier Detection Algorithm Based on Fast Density Peak Clustering Outlier Factor
    Zhang, ZhongPing
    Li, Sen
    Liu, WeiXiong
    Wang, Ying
    Li, Daisy Xin
    [J]. INTERNATIONAL JOURNAL OF DATA WAREHOUSING AND MINING, 2023, 19 (02)
  • [7] Location algorithm of transfer stations based on density peak and outlier detection
    Yan Shao-hong
    Niu Jia-yang
    Chen Tai-long
    Liu Qiu-tong
    Yang Cen
    Cheng Jia-qing
    Fu Zhi-zhen
    Li Jie
    [J]. Applied Intelligence, 2022, 52 : 13520 - 13532
  • [8] Location algorithm of transfer stations based on density peak and outlier detection
    Yan Shao-hong
    Niu Jia-yang
    Chen Tai-long
    Liu Qiu-tong
    Yang Cen
    Cheng Jia-qing
    Fu Zhi-zhen
    Li Jie
    [J]. APPLIED INTELLIGENCE, 2022, 52 (12) : 13520 - 13532
  • [9] Big Data Sampling Algorithm Based on Peak Detection
    Liu, Mengyu
    Wang, Yuhang
    Lin, Ruishi
    Wang, Shenhang
    Zheng, Wei
    [J]. PROCEEDINGS OF THE 33RD CHINESE CONTROL AND DECISION CONFERENCE (CCDC 2021), 2021, : 7573 - 7578
  • [10] A Big Data Online Cleaning Algorithm Based on Dynamic Outlier Detection
    Diao, Yinglong
    Liu, Ke-yan
    Meng, Xiaoli
    Ye, Xueshun
    He, Kaiyuan
    [J]. 2015 INTERNATIONAL CONFERENCE ON CYBER-ENABLED DISTRIBUTED COMPUTING AND KNOWLEDGE DISCOVERY, 2015, : 230 - 234