Investigation of out-of-distribution detection across various models and training methodologies

被引:0
|
作者
Kim, Byung Chun [1 ,3 ]
Kim, Byungro [2 ]
Hyun, Yoonsuk [2 ]
机构
[1] Inha Univ, Inst Appl Math, 100 Inha Ro, Incheon 22212, South Korea
[2] Inha Univ, Dept Math, 100 Inha Ro, Incheon 22212, South Korea
[3] SmartSocial, AI Lab, 140 Suyeonggangbyeon-Daero, Pusan 48058, South Korea
基金
新加坡国家研究基金会;
关键词
Out-of-distribution detection; Machine learning; Label smoothing; Image classification; Explainable AI;
D O I
10.1016/j.neunet.2024.106288
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Machine learning -based algorithms demonstrate impressive performance across numerous fields; however, they continue to suffer from certain limitations. Even sophisticated and precise algorithms often make erroneous predictions when implemented with datasets having different distributions compared to the training set. Out -ofdistribution (OOD) detection, which distinguishes data with different distributions from that of the training set, is a critical research area necessary to overcome these limitations and create more reliable algorithms. The OOD issue, particularly concerning image data, has been extensively studied. However, recently developed OOD methods do not fulfill the expectation that OOD performance will increase as the accuracy of in -distribution classification improves. Our research presents a comprehensive study on OOD detection performance across multiple models and training methodologies to verify this phenomenon. Specifically, we explore various pretrained models popular in the computer vision field with both old and new OOD detection methods. The experimental results highlight the performance disparity in existing OOD methods. Based on these observations, we introduce Trimmed Rank with Inverse softMax probability (TRIM), a remarkably simple yet effective method for model weights with newly developed training methods. The proposed method could serve as a potential tool for enhancing OOD detection performance owing to its promising results. The OOD performance of TRIM is highly compatible with the in -distribution accuracy model and may bridge the efforts on improving in -distribution accuracy to the ability to distinguish OOD data.
引用
收藏
页数:25
相关论文
共 50 条
  • [1] Latent Transformer Models for out-of-distribution detection
    Graham, Mark S.
    Tudosiu, Petru-Daniel
    Wright, Paul
    Pinaya, Walter Hugo Lopez
    Teikari, Petteri
    Patel, Ashay
    U-King-Im, Jean-Marie
    Mah, Yee H.
    Teo, James T.
    Jager, Hans Rolf
    Werring, David
    Rees, Geraint
    Nachev, Parashkev
    Ourselin, Sebastien
    Cardoso, M. Jorge
    [J]. MEDICAL IMAGE ANALYSIS, 2023, 90
  • [2] Deep Hybrid Models for Out-of-Distribution Detection
    Cao, Senqi
    Zhang, Zhongfei
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 4723 - 4733
  • [3] Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sources
    Zheng, Haotian
    Wang, Qizhou
    Fang, Zhen
    Xia, Xiaobo
    Liu, Feng
    Liu, Tongliang
    Han, Bo
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [4] Diffusion models for out-of-distribution detection in digital pathology
    Linmans, Jasper
    Raya, Gabriel
    van der Laak, Jeroen
    Litjens, Geert
    [J]. Medical Image Analysis, 2024, 93
  • [5] Diffusion models for out-of-distribution detection in digital pathology
    Linmans, Jasper
    Raya, Gabriel
    van der Laak, Jeroen
    Litjens, Geert
    [J]. MEDICAL IMAGE ANALYSIS, 2024, 93
  • [6] On the Learnability of Out-of-distribution Detection
    Fang, Zhen
    Li, Yixuan
    Liu, Feng
    Han, Bo
    Lu, Jie
    [J]. Journal of Machine Learning Research, 2024, 25
  • [7] Watermarking for Out-of-distribution Detection
    Wang, Qizhou
    Liu, Feng
    Zhang, Yonggang
    Zhang, Jing
    Gong, Chen
    Liu, Tongliang
    Han, Bo
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [8] Entropic Out-of-Distribution Detection
    Macedo, David
    Ren, Tsang Ing
    Zanchettin, Cleber
    Oliveira, Adriano L., I
    Ludermir, Teresa
    [J]. 2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2021,
  • [9] On the Learnability of Out-of-distribution Detection
    Fang, Zhen
    Li, Yixuan
    Liu, Feng
    Han, Bo
    Lu, Jie
    [J]. JOURNAL OF MACHINE LEARNING RESEARCH, 2024, 25
  • [10] Is Out-of-Distribution Detection Learnable?
    Fang, Zhen
    Li, Yixuan
    Lu, Jie
    Dong, Jiahua
    Han, Bo
    Liu, Feng
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,