Evaluation Metrics Research for Explainable Artificial Intelligence Global Methods Using Synthetic Data

被引:5
|
作者
Oblizanov, Alexandr [1 ]
Shevskaya, Natalya [1 ]
Kazak, Anatoliy [2 ]
Rudenko, Marina [3 ]
Dorofeeva, Anna [2 ]
机构
[1] St Petersburg Electrotech Univ Leti, Fac Comp Sci & Technol, St Petersburg 197376, Russia
[2] VI Vernadsky Crimean Fed Univ, Humanitarian Pedag Acad, Simferopol 295007, Russia
[3] VI Vernadsky Crimean Fed Univ, Inst Phys & Technol, Simferopol 295007, Russia
关键词
explainable artificial intelligence; XAI; explanation metrics; synthetic data; SMOTE;
D O I
10.3390/asi6010026
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In recent years, artificial intelligence technologies have been developing more and more rapidly, and a lot of research is aimed at solving the problem of explainable artificial intelligence. Various XAI methods are being developed to allow the user to understand the logic of how machine learning models work, and in order to compare the methods, it is necessary to evaluate them. The paper analyzes various approaches to the evaluation of XAI methods, defines the requirements for the evaluation system and suggests metrics to determine the various technical characteristics of the methods. A study was conducted, using these metrics, which determined the degradation in the explanation quality of the SHAP and LIME methods with increasing correlation in the input data. Recommendations are also given for further research in the field of practical implementation of metrics, expanding the scope of their use.
引用
收藏
页数:13
相关论文
共 50 条
  • [21] Mixing Approach for Text Data Augmentation Based on an Ensemble of Explainable Artificial Intelligence Methods
    Yu, Jinyi
    Choi, Jinhae
    Lee, Younghoon
    NEURAL PROCESSING LETTERS, 2023, 55 (02) : 1741 - 1757
  • [22] Discovering Themes in Deep Brain Stimulation Research Using Explainable Artificial Intelligence
    Allen, Ben
    BIOMEDICINES, 2023, 11 (03)
  • [23] Peering Into the Black Box of Artificial Intelligence: Evaluation Metrics of Machine Learning Methods
    Handelman, Guy S.
    Kok, Hong Kuan
    Chandra, Ronil V.
    Razavi, Amir H.
    Huang, Shiwei
    Brooks, Mark
    Lee, Michael J.
    Asadi, Hamed
    AMERICAN JOURNAL OF ROENTGENOLOGY, 2019, 212 (01) : 38 - 43
  • [24] A Bibliometric Analysis of the Explainable Artificial Intelligence Research Field
    Alonso, Jose M.
    Castiello, Ciro
    Mencar, Corrado
    INFORMATION PROCESSING AND MANAGEMENT OF UNCERTAINTY IN KNOWLEDGE-BASED SYSTEMS: THEORY AND FOUNDATIONS, IPMU 2018, PT I, 2018, 853 : 3 - 15
  • [25] Research on Explainable Artificial Intelligence Techniques: An User Perspective
    Daudt, Fabio
    Cinalli, Daniel
    Garcia, Ana Cristina B.
    PROCEEDINGS OF THE 2021 IEEE 24TH INTERNATIONAL CONFERENCE ON COMPUTER SUPPORTED COOPERATIVE WORK IN DESIGN (CSCWD), 2021, : 144 - 149
  • [26] The Review of Studies on Explainable Artificial Intelligence in Educational Research
    Turkmen, Gamze
    JOURNAL OF EDUCATIONAL COMPUTING RESEARCH, 2025, 63 (02) : 277 - 310
  • [27] Explainable Artificial Intelligence for Data Science on Customer Churn
    Leung, Carson K.
    Pazdor, Adam G. M.
    Souza, Joglas
    2021 IEEE 8TH INTERNATIONAL CONFERENCE ON DATA SCIENCE AND ADVANCED ANALYTICS (DSAA), 2021,
  • [28] An eXplainable Artificial Intelligence Methodology on Big Data Architecture
    La Gatta, Valerio
    Moscato, Vincenzo
    Postiglione, Marco
    Sperli, Giancarlo
    COGNITIVE COMPUTATION, 2024, 16 (05) : 2642 - 2659
  • [29] Explainable Artificial Intelligence for Crowd Forecasting Using Global Ensemble Echo State Networks
    Samarajeewa, Chamod
    De Silva, Daswin
    Manic, Milos
    Mills, Nishan
    Rathnayaka, Prabod
    Jennings, Andrew
    IEEE OPEN JOURNAL OF THE INDUSTRIAL ELECTRONICS SOCIETY, 2024, 5 : 415 - 427
  • [30] Notions of explainability and evaluation approaches for explainable artificial intelligence
    Vilone, Giulia
    Longo, Luca
    INFORMATION FUSION, 2021, 76 : 89 - 106