A novel approach for anomaly detection in data streams: Fuzzy-statistical detection mode

被引:4
|
作者
Li, Fenghuan [1 ]
Zheng, Dequan [1 ]
Zhao, Tiejun [1 ]
Pedrycz, Witold [2 ,3 ]
机构
[1] Harbin Inst Technol, MOE MS Key Lab Nat Language Proc & Speech, Harbin 150006, Heilongjiang, Peoples R China
[2] Univ Alberta, Dept Elect & Comp Engn, Edmonton, AB T6G 2G7, Canada
[3] Polish Acad Sci, Syst Res Inst, PL-01447 Warsaw, Poland
基金
中国国家自然科学基金;
关键词
Anomaly detection; statistical test; self-adaptive; fuzzy set theory; unsupervised; ARRHYTHMIA DETECTION; INTRUSION DETECTION; OUTLIER DETECTION; NEURAL-NETWORK; SYSTEM; ALGORITHMS; DIAGNOSIS; REPRESENTATION;
D O I
10.3233/IFS-151910
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
subsequences that exhibit departures from normal state of operation. In this paper, to solve the problems of unknown data distribution, control limit determination, multiple parameters, training data and fuzziness of 'anomaly', a self-adaptive and unsupervised model is developed for finding anomalies in data streams. A salient feature is a synergistic combination of both statistical and fuzzy set-based techniques. Anomaly detection problem is viewed as a certain statistical hypothesis testing which is realized in an unsupervised mode. At the same time, 'anomaly' is a much more complex concept and as such can be described with fuzzy set theory. Fuzzy sets bring a facet of robustness to the overall scheme and play an important role in the successive step of hypothesis testing. Because of the fuzzification, parameters determination is self-adaptive and no parameter needs to be specified by the user, what's more, there is no need to consider the data distribution in statistical hypothesis testing in this paper. The approach is validated with a number of experiments, which help to quantify the performance of constructed algorithm.
引用
收藏
页码:2611 / 2622
页数:12
相关论文
共 50 条
  • [31] From Anomaly Detection to Rumour Detection using Data Streams of Social Platforms
    Nguyen Thanh Tam
    Weidlich, Matthias
    Zheng, Bolong
    Yin, Hongzhi
    Nguyen Quoc Viet Hung
    Stantic, Bela
    [J]. PROCEEDINGS OF THE VLDB ENDOWMENT, 2019, 12 (09): : 1016 - 1029
  • [32] An immuno-fuzzy approach to anomaly detection
    Gómez, J
    González, F
    Dasgupta, D
    [J]. PROCEEDINGS OF THE 12TH IEEE INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS, VOLS 1 AND 2, 2003, : 1219 - 1224
  • [33] A Fuzzy Inspired Approach to Seismic Anomaly Detection
    Christodoulou, Vyron
    Bi, Yaxin
    Zhao, Guoze
    [J]. KNOWLEDGE SCIENCE, ENGINEERING AND MANAGEMENT, KSEM 2015, 2015, 9403 : 575 - 587
  • [34] Fuzzy Anomaly Detection in Monitoring Sensor Data
    Rabatel, Julien
    Bringay, Sandra
    Poncelet, Pascal
    [J]. 2010 IEEE INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS (FUZZ-IEEE 2010), 2010,
  • [35] A User Behavior Anomaly Detection Approach based on Sequence Mining over Data Streams
    Zhou, Yong
    Wang, Yijie
    Ma, Xingkong
    [J]. 2016 17TH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED COMPUTING, APPLICATIONS AND TECHNOLOGIES (PDCAT), 2016, : 376 - 381
  • [36] EADetection: An efficient and accurate sequential behavior anomaly detection approach over data streams
    Cheng, Li
    Wang, Yijie
    Zhou, Yong
    Ma, Xingkong
    [J]. INTERNATIONAL JOURNAL OF DISTRIBUTED SENSOR NETWORKS, 2018, 14 (10)
  • [37] Anomaly Detection in Catalog Streams
    Yang, Chen
    Du, Zhihui
    Meng, Xiaofeng
    Zhang, Xukang
    Hao, Xinli
    Bader, David A.
    [J]. IEEE TRANSACTIONS ON BIG DATA, 2023, 9 (01) : 294 - 311
  • [38] Visualization of Data Cubes for Anomaly Detection in Network Traffic Data Streams
    Ahlers, Volker
    Laue, Tim
    Wellermann, Nils
    Heine, Felix
    [J]. PROCEEDINGS OF THE THE 11TH IEEE INTERNATIONAL CONFERENCE ON INTELLIGENT DATA ACQUISITION AND ADVANCED COMPUTING SYSTEMS: TECHNOLOGY AND APPLICATIONS (IDAACS'2021), VOL 1, 2021, : 272 - 277
  • [39] Adaptive fuzzy-statistical decision model to grade sensor data
    Rokonuzzaman, M
    Gosine, RG
    [J]. 1997 CANADIAN CONFERENCE ON ELECTRICAL AND COMPUTER ENGINEERING, CONFERENCE PROCEEDINGS, VOLS I AND II: ENGINEERING INNOVATION: VOYAGE OF DISCOVERY, 1997, : 773 - 776
  • [40] MAG: A Novel Approach for Effective Anomaly Detection in Spacecraft Telemetry Data
    Yu, Bing
    Yu, Yang
    Xu, Jiakai
    Xiang, Gang
    Yang, Zhiming
    [J]. IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, 2024, 20 (03) : 3891 - 3899