Explainable Deep-Learning-Based Diagnosis of Alzheimer's Disease Using Multimodal Input Fusion of PET and MRI Images

被引:15
|
作者
Odusami, Modupe [1 ]
Maskeliunas, Rytis [1 ]
Damasevicius, Robertas [2 ]
Misra, Sanjay [3 ]
机构
[1] Kaunas Univ Technol, Dept Multimedia Engn, Kaunas, Lithuania
[2] Silesian Tech Univ, Fac Appl Math, Gliwice, Poland
[3] Inst Energy Technol, Halden, Norway
关键词
Alzheimer's disease; Feature fusion; MRI; PET; Heuristic methods; Deep learning; BIOMARKERS;
D O I
10.1007/s40846-023-00801-3
中图分类号
R318 [生物医学工程];
学科分类号
0831 ;
摘要
PurposeAlzheimer's disease (AD) is a progressive, incurable human brain illness that impairs reasoning and retention as well as recall. Detecting AD in its preliminary stages before clinical manifestations is crucial for timely treatment. Magnetic Resonance Imaging (MRI) provides valuable insights into brain abnormalities by measuring the decrease in brain volume expressly in the mesial temporal cortex and other regions of the brain, while Positron Emission Tomography (PET) measures the decrease of glucose concentration in the temporoparietal association cortex. When these data are combined, the performance of AD diagnostic methods could be improved. However, these data are heterogeneous and there is a need for an effective model that will harness the information from both data for the accurate prediction of AD.MethodsTo this end, we present a novel heuristic early feature fusion framework that performs the concatenation of PET and MRI images, while a modified Resnet18 deep learning architecture is trained simultaneously on the two datasets. The innovative 3-in-channel approach is used to learn the most descriptive features of fused PET and MRI images for effective binary classification of AD.ResultsThe experimental results show that the proposed model achieved a classification accuracy of 73.90% on the ADNI database. Then, we provide an Explainable Artificial Intelligence (XAI) model, allowing us to explain the results.ConclusionOur proposed model could learn latent representations of multimodal data even in the presence of heterogeneity data; hence, the proposed model partially solved the issue with the heterogeneity of the MRI and PET data.
引用
收藏
页码:291 / 302
页数:12
相关论文
共 50 条
  • [1] Explainable Deep-Learning-Based Diagnosis of Alzheimer’s Disease Using Multimodal Input Fusion of PET and MRI Images
    Modupe Odusami
    Rytis Maskeliūnas
    Robertas Damaševičius
    Sanjay Misra
    [J]. Journal of Medical and Biological Engineering, 2023, 43 : 291 - 302
  • [2] Input Agnostic Deep Learning for Alzheimer's Disease Classification Using Multimodal MRI Images
    Massalimova, Aidana
    Varol, Huseyin Atakan
    [J]. 2021 43RD ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE & BIOLOGY SOCIETY (EMBC), 2021, : 2875 - 2878
  • [3] Deep learning based diagnosis of Alzheimer's disease using FDG-PET images
    Kishore, Nand
    Goel, Neelam
    [J]. NEUROSCIENCE LETTERS, 2023, 817
  • [4] An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis
    Song, Juan
    Zheng, Jian
    Li, Ping
    Lu, Xiaoyuan
    Zhu, Guangming
    Shen, Peiyi
    [J]. FRONTIERS IN DIGITAL HEALTH, 2021, 3
  • [5] Multimodal EEG, MRI and PET Data Fusion for Alzheimer's Disease Diagnosis
    Polikar, Robi
    Tilley, Christopher
    Hillis, Brendan
    Clark, Chris M.
    [J]. 2010 ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC), 2010, : 6058 - 6061
  • [6] TRANSFORMER-BASED MULTIMODAL FUSION FOR EARLY DIAGNOSIS OF ALZHEIMER'S DISEASE USING STRUCTURAL MRI AND PET
    Zhang, Yuanwang
    Sun, Kaicong
    Liu, Yuxiao
    Shen, Dinggang
    [J]. 2023 IEEE 20TH INTERNATIONAL SYMPOSIUM ON BIOMEDICAL IMAGING, ISBI, 2023,
  • [7] Assisted Diagnosis of Alzheimer's Disease Based on Deep Learning and Multimodal Feature Fusion
    Wang, Yu
    Liu, Xi
    Yu, Chongchong
    [J]. COMPLEXITY, 2021, 2021
  • [8] Explainable Deep CNNs for MRI-Based Diagnosis of Alzheimer's Disease
    Nigri, Eduardo
    Ziviani, Nivio
    Cappabianco, Fabio
    Antunes, Augusto
    Veloso, Adriano
    [J]. 2020 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2020,
  • [9] Multimodal Fusion-Based Deep Learning Network for Effective Diagnosis of Alzheimer's Disease
    Dwivedi, Shubham
    Goel, Tripti
    Tanveer, M.
    Murugan, R.
    Sharma, Rahul
    [J]. IEEE MULTIMEDIA, 2022, 29 (02) : 45 - 55
  • [10] Deep learning and multimodal feature fusion for the aided diagnosis of Alzheimer's disease
    Jia, Hongfei
    Lao, Huan
    [J]. NEURAL COMPUTING & APPLICATIONS, 2022, 34 (22): : 19585 - 19598