A novel approach for anomaly detection in data streams: Fuzzy-statistical detection mode

被引:4
|
作者
Li, Fenghuan [1 ]
Zheng, Dequan [1 ]
Zhao, Tiejun [1 ]
Pedrycz, Witold [2 ,3 ]
机构
[1] Harbin Inst Technol, MOE MS Key Lab Nat Language Proc & Speech, Harbin 150006, Heilongjiang, Peoples R China
[2] Univ Alberta, Dept Elect & Comp Engn, Edmonton, AB T6G 2G7, Canada
[3] Polish Acad Sci, Syst Res Inst, PL-01447 Warsaw, Poland
基金
中国国家自然科学基金;
关键词
Anomaly detection; statistical test; self-adaptive; fuzzy set theory; unsupervised; ARRHYTHMIA DETECTION; INTRUSION DETECTION; OUTLIER DETECTION; NEURAL-NETWORK; SYSTEM; ALGORITHMS; DIAGNOSIS; REPRESENTATION;
D O I
10.3233/IFS-151910
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
subsequences that exhibit departures from normal state of operation. In this paper, to solve the problems of unknown data distribution, control limit determination, multiple parameters, training data and fuzziness of 'anomaly', a self-adaptive and unsupervised model is developed for finding anomalies in data streams. A salient feature is a synergistic combination of both statistical and fuzzy set-based techniques. Anomaly detection problem is viewed as a certain statistical hypothesis testing which is realized in an unsupervised mode. At the same time, 'anomaly' is a much more complex concept and as such can be described with fuzzy set theory. Fuzzy sets bring a facet of robustness to the overall scheme and play an important role in the successive step of hypothesis testing. Because of the fuzzification, parameters determination is self-adaptive and no parameter needs to be specified by the user, what's more, there is no need to consider the data distribution in statistical hypothesis testing in this paper. The approach is validated with a number of experiments, which help to quantify the performance of constructed algorithm.
引用
收藏
页码:2611 / 2622
页数:12
相关论文
共 50 条
  • [1] Evolving Fuzzy Rules for Anomaly Detection in Data Streams
    Moshtaghi, Masud
    Bezdek, James C.
    Leckie, Christopher
    Karunasekera, Shanika
    Palaniswami, Marimuthu
    [J]. IEEE TRANSACTIONS ON FUZZY SYSTEMS, 2015, 23 (03) : 688 - 700
  • [2] Anomaly Detection in Data Streams using Fuzzy Logic
    Khan, Muhammad Umair
    [J]. 2009 INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGIES, 2009, : 126 - 133
  • [3] Anomaly detection for smartphone data streams
    Mirsky, Yisroel
    Shabtai, Asaf
    Shapira, Bracha
    Elovici, Yuval
    Rokach, Lior
    [J]. PERVASIVE AND MOBILE COMPUTING, 2017, 35 : 83 - 107
  • [4] Anomaly Pattern Detection on Data Streams
    Park, Cheong Hee
    [J]. 2018 IEEE INTERNATIONAL CONFERENCE ON BIG DATA AND SMART COMPUTING (BIGCOMP), 2018, : 689 - 692
  • [5] Anomaly Detection Guidelines for Data Streams in Big Data
    Rana, Annie Ibrahim
    Estrada, Giovani
    Sole, Marc
    Muntes, Victor
    [J]. 2016 3RD INTERNATIONAL CONFERENCE ON SOFT COMPUTING & MACHINE INTELLIGENCE (ISCMI 2016), 2016, : 94 - 98
  • [6] Outlier and anomaly pattern detection on data streams
    Cheong Hee Park
    [J]. The Journal of Supercomputing, 2019, 75 : 6118 - 6128
  • [7] Adaptive Anomaly Detection on Network Data Streams
    Riddle-Workman, Elizabeth
    Evangelou, Marina
    Adams, Niall M.
    [J]. 2018 IEEE INTERNATIONAL CONFERENCE ON INTELLIGENCE AND SECURITY INFORMATICS (ISI), 2018, : 19 - 24
  • [8] Review of Anomaly Detection Algorithms for Data Streams
    Lu, Tianyuan
    Wang, Lei
    Zhao, Xiaoyong
    [J]. APPLIED SCIENCES-BASEL, 2023, 13 (10):
  • [9] Anomaly Detection on Data Streams for Smart Agriculture
    Moso, Juliet Chebet
    Cormier, Stephane
    de Runz, Cyril
    Fouchal, Hacene
    Wandeto, John Mwangi
    [J]. AGRICULTURE-BASEL, 2021, 11 (11):
  • [10] OHODIN - Online Anomaly Detection for Data Streams
    Gruhl, Christian
    Tomforde, Sven
    [J]. 2021 IEEE INTERNATIONAL CONFERENCE ON AUTONOMIC COMPUTING AND SELF-ORGANIZING SYSTEMS COMPANION (ACSOS-C 2021), 2021, : 193 - 197