Evaluation Metrics Research for Explainable Artificial Intelligence Global Methods Using Synthetic Data

被引:5
|
作者
Oblizanov, Alexandr [1 ]
Shevskaya, Natalya [1 ]
Kazak, Anatoliy [2 ]
Rudenko, Marina [3 ]
Dorofeeva, Anna [2 ]
机构
[1] St Petersburg Electrotech Univ Leti, Fac Comp Sci & Technol, St Petersburg 197376, Russia
[2] VI Vernadsky Crimean Fed Univ, Humanitarian Pedag Acad, Simferopol 295007, Russia
[3] VI Vernadsky Crimean Fed Univ, Inst Phys & Technol, Simferopol 295007, Russia
关键词
explainable artificial intelligence; XAI; explanation metrics; synthetic data; SMOTE;
D O I
10.3390/asi6010026
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In recent years, artificial intelligence technologies have been developing more and more rapidly, and a lot of research is aimed at solving the problem of explainable artificial intelligence. Various XAI methods are being developed to allow the user to understand the logic of how machine learning models work, and in order to compare the methods, it is necessary to evaluate them. The paper analyzes various approaches to the evaluation of XAI methods, defines the requirements for the evaluation system and suggests metrics to determine the various technical characteristics of the methods. A study was conducted, using these metrics, which determined the degradation in the explanation quality of the SHAP and LIME methods with increasing correlation in the input data. Recommendations are also given for further research in the field of practical implementation of metrics, expanding the scope of their use.
引用
收藏
页数:13
相关论文
共 50 条
  • [31] A Trustworthy View on Explainable Artificial Intelligence Method Evaluation
    Li, Ding
    Liu, Yan
    Huang, Jun
    Wang, Zerui
    COMPUTER, 2023, 56 (04) : 50 - 60
  • [32] On evaluation metrics for medical applications of artificial intelligence
    Steven A. Hicks
    Inga Strümke
    Vajira Thambawita
    Malek Hammou
    Michael A. Riegler
    Pål Halvorsen
    Sravanthi Parasa
    Scientific Reports, 12
  • [33] On evaluation metrics for medical applications of artificial intelligence
    Hicks, Steven A.
    Struemke, Inga
    Thambawita, Vajira
    Hammou, Malek
    Riegler, Michael A.
    Halvorsen, Pal
    Parasa, Sravanthi
    SCIENTIFIC REPORTS, 2022, 12 (01)
  • [34] Leveraging Artificial Intelligence and Synthetic Data Derivatives for Spine Surgery Research
    Greenberg, Jacob K.
    Landman, Joshua M.
    Kelly, Michael P.
    Pennicooke, Brenton H.
    Molina, Camilo A.
    Foraker, Randi E.
    Ray, Wilson Z.
    GLOBAL SPINE JOURNAL, 2023, 13 (08) : 2409 - 2421
  • [35] A New Perspective on hvaluation Methods for Explainable Artificial Intelligence (XAI)
    Speith, Timo
    Langer, Markus
    2023 IEEE 31ST INTERNATIONAL REQUIREMENTS ENGINEERING CONFERENCE WORKSHOPS, REW, 2023, : 325 - 331
  • [36] EXplainable Artificial Intelligence (XAI)-From Theory to Methods and Applications
    Ortigossa, Evandro S.
    Goncalves, Thales
    Nonato, Luis Gustavo
    IEEE ACCESS, 2024, 12 : 80799 - 80846
  • [37] Explainable Artificial Intelligence Methods in Combating Pandemics: A Systematic Review
    Giuste, Felipe
    Shi, Wenqi
    Zhu, Yuanda
    Naren, Tarun
    Isgut, Monica
    Sha, Ying
    Tong, Li
    Gupte, Mitali
    Wang, May D.
    IEEE REVIEWS IN BIOMEDICAL ENGINEERING, 2023, 16 : 5 - 21
  • [38] Explainable Artificial Intelligence for Business and Economics: Methods, Applications and Challenges
    Lyu, Qi
    Wu, Shaomin
    EXPERT SYSTEMS, 2025, 42 (04)
  • [39] Classification of Explainable Artificial Intelligence Methods through Their Output Formats
    Vilone, Giulia
    Longo, Luca
    MACHINE LEARNING AND KNOWLEDGE EXTRACTION, 2021, 3 (03): : 615 - 661
  • [40] Explainable Artificial Intelligence Methods in Combating Pandemics: A Systematic Review
    Giuste, Felipe
    Shi, Wenqi
    Zhu, Yuanda
    Naren, Tarun
    Isgut, Monica
    Sha, Ying
    Tong, Li
    Gupte, Mitali
    Wang, May D. D.
    IEEE REVIEWS IN BIOMEDICAL ENGINEERING, 2023, 16 : 5 - 21