BTDNet: A Multi-Modal Approach for Brain Tumor Radiogenomic Classification

被引:1
|
作者
Kollias, Dimitrios [1 ]
Vendal, Karanjot [1 ]
Gadhavi, Priyankaben [1 ]
Russom, Solomon [1 ]
机构
[1] Queen Mary Univ London, Sch Elect Engn & Comp Sci, London E1 4NS, England
来源
APPLIED SCIENCES-BASEL | 2023年 / 13卷 / 21期
关键词
RSNA-ASNR-MICCAI BraTS 2021 Challenge; brain tumor radiogenomic classification; prediction of MGMT promoter methylation status; BTDNet; multimodal approach; routing; mask layer; MixAugment; multi-class focal loss; volume-level annotations; variable-length data;
D O I
10.3390/app132111984
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Brain tumors pose significant health challenges worldwide, with glioblastoma being one of the most aggressive forms. The accurate determination of the O6-methylguanine-DNA methyltransferase (MGMT) promoter methylation status is crucial for personalized treatment strategies. However, traditional methods are labor-intensive and time-consuming. This paper proposes a novel multi-modal approach, BTDNet, that leverages multi-parametric MRI scans, including FLAIR, T1w, T1wCE, and T2 3D volumes, to predict the MGMT promoter methylation status. BTDNet's main contribution involves addressing two main challenges: the variable volume lengths (i.e., each volume consists of a different number of slices) and the volume-level annotations (i.e., the whole 3D volume is annotated and not the independent slices that it consists of). BTDNet consists of four components: (i) data augmentation (which performs geometric transformations, convex combinations of data pairs, and test-time data augmentation); (ii) 3D analysis (which performs global analysis through a CNN-RNN); (iii) routing (which contains a mask layer that handles variable input feature lengths); and (iv) modality fusion (which effectively enhances data representation, reduces ambiguities, and mitigates data scarcity). The proposed method outperformed state-of-the-art methods in the RSNA-ASNR-MICCAI BraTS 2021 Challenge by at least 3.3% in terms of the F1 score, offering a promising avenue for enhancing brain tumor diagnosis and treatment.
引用
收藏
页数:15
相关论文
共 50 条
  • [31] Multi-modal long document classification based on Hierarchical Prompt and Multi-modal Transformer
    Liu, Tengfei
    Hu, Yongli
    Gao, Junbin
    Wang, Jiapu
    Sun, Yanfeng
    Yin, Baocai
    NEURAL NETWORKS, 2024, 176
  • [32] TumorGAN: A Multi-Modal Data Augmentation Framework for Brain Tumor Segmentation
    Li, Qingyun
    Yu, Zhibin
    Wang, Yubo
    Zheng, Haiyong
    SENSORS, 2020, 20 (15) : 1 - 16
  • [33] Deep fusion of multi-modal features for brain tumor image segmentation
    Zhang, Guying
    Zhou, Jia
    He, Guanghua
    Zhu, Hancan
    HELIYON, 2023, 9 (08)
  • [34] Multi-modal Brain Tumor Segmentation Utilizing Convolutional Neural Networks
    Jakab, Marek
    Stevuliak, Marek
    Benesova, Wanda
    TWELFTH INTERNATIONAL CONFERENCE ON MACHINE VISION (ICMV 2019), 2020, 11433
  • [35] Multi-modal Image Classification Using Low-Dimensional Texture Features for Genomic Brain Tumor Recognition
    Alberts, Esther
    Tetteh, Giles
    Trebeschi, Stefano
    Bieth, Marie
    Valentinitsch, Alexander
    Wiestler, Benedikt
    Zimmer, Claus
    Menze, Bjoern H.
    GRAPHS IN BIOMEDICAL IMAGE ANALYSIS, COMPUTATIONAL ANATOMY AND IMAGING GENETICS, 2017, 10551 : 201 - 209
  • [36] Brain Tumor Radiogenomic Classification Using Deep Learning Algorithms
    Abdullah, Azian Azamimi
    Zaharuddin, Nur Balqis Hanum
    Mohammad, Nur Farahiyah
    Mohamed, Latifah
    INTELLIGENT MANUFACTURING AND MECHATRONICS, SIMM 2023, 2024, : 771 - 788
  • [37] In vivo characterization of tumor and tumor vascular network using multi-modal imaging approach
    Kalchenko, Vyacheslav
    Madar-Balakirski, Noa
    Meglinski, Igor
    Harmelin, Alon
    JOURNAL OF BIOPHOTONICS, 2011, 4 (09) : 645 - 649
  • [38] Cross-Modal Retrieval Augmentation for Multi-Modal Classification
    Gur, Shir
    Neverova, Natalia
    Stauffer, Chris
    Lim, Ser-Nam
    Kiela, Douwe
    Reiter, Austin
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, EMNLP 2021, 2021, : 111 - 123
  • [39] Automated brain tumor segmentation on multi-modal MR image using SegNet
    Salma Alqazzaz
    Xianfang Sun
    Xin Yang
    Len Nokes
    Computational Visual Media, 2019, 5 : 209 - 219
  • [40] Multi-modal Fusion Brain Tumor Detection Method Based on Deep Learning
    Yao Hong-ge
    Shen Xin-xia
    Li Yu
    Yu Jun
    Lei Song-ze
    ACTA PHOTONICA SINICA, 2019, 48 (07)