Anomaly Detection Under Contaminated Data With Contamination-Immune Bidirectional GANs

被引:0
|
作者
Su Q. [1 ]
Tian B. [1 ]
Wan H. [1 ]
Yin J. [2 ]
机构
[1] School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou
[2] Scool of Artificial Intelligence, Sun Yat-sen University
关键词
Anomaly detection; Contamination; data contamination; generative adversarial networks (GAN); generative models; Medical diagnostic imaging; outlier detection; Probability distribution; Prototypes; Support vector machines; Training;
D O I
10.1109/TKDE.2024.3404027
中图分类号
学科分类号
摘要
Anomaly detection aims to detect instances that deviate significantly from the majority. Due to the difficulties of collecting a large amount of anomalies in practice, existing methods generally assume the availability of a clean normal dataset and leverage it to detect anomalies by characterizing the normality of normal samples. However, for many application scenarios, collecting a normal dataset that is sufficiently clean is not easy. What is often observed is that a small amount of anomalies are often falsely mixed into the normal dataset, resulting in a contaminated dataset. Obviously, the contamination in the normal dataset could significantly compromise the model's ability to detect anomalies. To alleviate this issue, two contamination-immune bidirectional generative adversarial networks (BiGAN) are developed, which can learn the probability distribution of normal samples from a contaminated dataset under some mild conditions. Rigorous proofs are provided to guarantee the theoretical correctness of the proposed models. Thanks to the removing of negative influences from the contamination samples, the proposed contamination-immune models can thus be applied to detect anomalies accurately for the scenarios with contaminated datasets. Extensive experimental results show that the proposed method outperforms the current state-of-the-art (SOTA) ones significantly under the scenarios with contaminated training datasets. IEEE
引用
收藏
页码:1 / 16
页数:15
相关论文
共 29 条
  • [1] An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contamination
    Kim, Minkyung
    Yu, Jongmin
    Kim, Junsik
    Oh, Tae-Hyun
    Choi, Jun Kyun
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, : 1 - 13
  • [2] Anomaly Detection Using GANs for Visual Inspection in Noisy Training Data
    Kimura, Masanari
    Yanagihara, Takashi
    COMPUTER VISION - ACCV 2018 WORKSHOPS, 2019, 11367 : 373 - 385
  • [3] Latent Outlier Exposure for Anomaly Detection with Contaminated Data
    Qiu, Chen
    Li, Aodong
    Kloft, Marius
    Rudolph, Maja
    Mandt, Stephan
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162, 2022,
  • [4] Coupled IGMM-GANs with Applications to Anomaly Detection in Human Mobility Data
    Smolyak, Daniel
    Gray, Kathryn
    Badirli, Sarkhan
    Mohler, George
    ACM TRANSACTIONS ON SPATIAL ALGORITHMS AND SYSTEMS, 2020, 6 (04)
  • [5] Anomaly detection of high-dimensional data based on Ensemble GANs with Dropout
    Chen, Wanghu
    Yao, Jilong
    Zhou, Meilin
    Li, Jing
    Shen, Mengyang
    2022 IEEE/ACM INTERNATIONAL CONFERENCE ON BIG DATA COMPUTING, APPLICATIONS AND TECHNOLOGIES, BDCAT, 2022, : 180 - 181
  • [6] Efficient anomaly detection on sampled data streams with contaminated phase I data
    El Sibai, Rayane
    Abdo, Jacques Bou
    Abou Jaoude, Chady
    Demerjian, Jacques
    Assaker, Joseph
    Makhoul, Abdallah
    INTERNET TECHNOLOGY LETTERS, 2020, 3 (05)
  • [7] Comparative Evaluation of VAEs, VAE-GANs and AAEs for Anomaly Detection in Network Intrusion Data
    Mohamed, Mahmoud
    EMITTER-INTERNATIONAL JOURNAL OF ENGINEERING TECHNOLOGY, 2023, 11 (02) : 160 - 173
  • [8] Sensor Data Reconstruction and Anomaly Detection using Bidirectional Recurrent Neural Network
    Jeong, Seongwoon
    Ferguson, Max
    Law, Kincho H.
    SENSORS AND SMART STRUCTURES TECHNOLOGIES FOR CIVIL, MECHANICAL, AND AEROSPACE SYSTEMS 2019, 2019, 10970
  • [9] Anomaly Detection: Under the [data] hood in Smart Cars
    Quader, Faisal
    Janeja, Vandana P.
    2019 IEEE INTERNATIONAL CONFERENCE ON SMART COMPUTING (SMARTCOMP 2019), 2019, : 126 - 131
  • [10] A novel anomaly detection algorithm for sensor data under uncertainty
    Raihan Ul Islam
    Mohammad Shahadat Hossain
    Karl Andersson
    Soft Computing, 2018, 22 : 1623 - 1639