BathNet: A network to classification of fundus and contrast images using label transfer and multi-branch transformer

被引:0
|
作者
Wang, Yaqi [1 ]
Xv, Zihao [2 ]
Wang, Yizhen [2 ]
Jin, Kai [3 ]
Gao, Zhiyuan [3 ]
Ke, Yiran [2 ]
Wu, Chengyu [4 ]
Chen, Xiaodiao [1 ,2 ]
Chen, Dechao [2 ]
Ye, Juan [3 ]
Jia, Gangyong [2 ]
机构
[1] Commun Univ Zhejiang, Coll Media Engn, Hangzhou 310018, Peoples R China
[2] Hangzhou Dianzi Univ, Sch Comp Sci & Technol, Hangzhou 310018, Peoples R China
[3] Zhejiang Univ, Coll Med, Dept Ophthalmol, Affiliated Hosp 2, Hangzhou, Peoples R China
[4] Shandong Univ, Dept Mech Elect & Informat Engn, Weihai 264209, Peoples R China
基金
中国国家自然科学基金;
关键词
Diabetic retinopathy; Self -calibrated convolutional; Label transfer; Multi -branch structure; Colorful fundus image; Fluorescein angiography fundus image; DIABETIC-RETINOPATHY; DISEASE;
D O I
10.1016/j.bspc.2024.106409
中图分类号
R318 [生物医学工程];
学科分类号
0831 ;
摘要
Diabetic retinopathy (DRP) is commonly caused by the complications of diabetes and, if ignored and untreated, may lead to irreversible blindness in adults. Medical experts mainly diagnose DRP by its disease features, such as microangioma (MA), exudate (EX), and hemorrhage (HE). However, there are limitations in the diagnostic effect due to the small disease features lesions and inconspicuous feature areas of DRP. In this paper, we propose a self -calibrated convolutional network, BathNet, for fine-grained classification, to assist in the diagnosis of three disease features, MA, EX, and HE. In this network we replace the basic convolutional blocks with self -calibrated convolutional blocks so that the internal communication can be extended the field of view of each convolutional layer to extract feature information efficiently. Then, in the multi -branch structure, we obtain local features by using the CNN structure with self -calibrated convolution blocks, and global features by using the progressive transformer structure to obtain the remote dependencies of the feature mapping. Finally, the relatively important feature channels are filtered out by the branch fusion module. In addition, many DRP-related disease features (e.g., EX) must be observed by the fluorescein angiography fundus image (FAF) but not by the colorful fundus image (CFP), so we propose a method for label migration across modalities by corresponding feature labels on FAF images to CFP images as network inputs. The experiment showed that the prediction results of CFP images after label migration were similar to FAF images. Comparing with baseline, our study BathNet achieves the best accuracy for MA, EX, and HE disease features, from 57.01%, 58.88%, and 64.49% to 78.5%, 71.96%, and 77.57%, respectively. In the future, patients can diagnose the disease features on FAF images with only one CFP image, which largely reduces the cost of disease detection.
引用
收藏
页数:12
相关论文
共 50 条
  • [1] Cross-attention multi-branch network for fundus diseases classification using SLO images
    Xie, Hai
    Zeng, Xianlu
    Lei, Haijun
    Du, Jie
    Wang, Jiantao
    Zhang, Guoming
    Cao, Jiuwen
    Wang, Tianfu
    Lei, Baiying
    [J]. MEDICAL IMAGE ANALYSIS, 2021, 71
  • [2] Multi-Branch Regression Network For Building Classification Using Remote Sensing Images
    Gui, Yuanyuan
    Li, Xiang
    Li, Wei
    Yue, Anzhi
    [J]. 2018 10TH IAPR WORKSHOP ON PATTERN RECOGNITION IN REMOTE SENSING (PRRS), 2018,
  • [3] Leadwise clustering multi-branch network for multi-label ECG classification
    Zhou, Feiyan
    Chen, Lingzhi
    [J]. MEDICAL ENGINEERING & PHYSICS, 2024, 130
  • [4] Multi-label classification of retinal diseases based on fundus images using Resnet and Transformer
    Zhao, Jiaqing
    Zhu, Jianfeng
    He, Jiangnan
    Cao, Guogang
    Dai, Cuixia
    [J]. MEDICAL & BIOLOGICAL ENGINEERING & COMPUTING, 2024,
  • [5] Multi-label classification of fundus images based on graph convolutional network
    Cheng, Yinlin
    Ma, Mengnan
    Li, Xingyu
    Zhou, Yi
    [J]. BMC MEDICAL INFORMATICS AND DECISION MAKING, 2021, 21 (SUPPL 2)
  • [6] Multi-label classification of fundus images with graph convolutional network and LightGBM
    Sun, Kai
    He, Mengjia
    Xu, Yao
    Wu, Qinying
    He, Zichun
    Li, Wang
    Liu, Hongying
    Pi, Xitian
    [J]. COMPUTERS IN BIOLOGY AND MEDICINE, 2022, 149
  • [7] Multi-label classification of fundus images based on graph convolutional network
    Yinlin Cheng
    Mengnan Ma
    Xingyu Li
    Yi Zhou
    [J]. BMC Medical Informatics and Decision Making, 21
  • [8] MultiTrans: Multi-branch transformer network for medical image segmentation
    Zhang, Yanhua
    Balestra, Gabriella
    Zhang, Ke
    Wang, Jingyu
    Rosati, Samanta
    Giannini, Valentina
    [J]. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE, 2024, 254
  • [9] Multi-branch fusion network for hyperspectral image classification
    Gao, Hongmin
    Yang, Yao
    Lei, Sheng
    Li, Chenming
    Zhou, Hui
    Qu, Xiaoyu
    [J]. KNOWLEDGE-BASED SYSTEMS, 2019, 167 : 11 - 25
  • [10] Multi-Label Classification of Fundus Images With EfficientNet
    Wang, Jing
    Yang, Liu
    Huo, Zhanqiang
    He, Weifeng
    Luo, Junwei
    [J]. IEEE ACCESS, 2020, 8 : 212499 - 212508