Improved Multi-modal Image Fusion with Attention and Dense Networks: Visual and Quantitative Evaluation

被引:0
|
作者
Banerjee, Ankan [1 ]
Patra, Dipti [1 ]
Roy, Pradipta [2 ]
机构
[1] Natl Inst Technol, Rourkela, India
[2] DRDO, Integrated Test Range, Candipur, India
关键词
image fusion; attention; human perception; Convolutional Block Attention Module;
D O I
10.1007/978-3-031-58535-7_20
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This article introduces a novel multi-modal image fusion approach based on Convolutional Block Attention Module and dense networks to enhance human perceptual quality and information content in the fused images. The proposed model preserves the edges of the infrared images and enhances the contrast of the visible image as a pre-processing part. Consequently, the use of Convolutional Block Attention Module has resulted in the extraction of more refined features from the source images. The visual results demonstrate that the fused images produced by the proposed method are visually superior to those generated by most standard fusion techniques. To substantiate the findings, quantitative analysis is conducted using various metrics. The proposed method exhibits the best Naturalness Image Quality Evaluator and Chen-Varshney metric values, which are human perception-based parameters. Moreover, the fused images exhibit the highest Standard Deviation value, signifying enhanced contrast. These results justify the proposed multi-modal image fusion technique outperforms standard methods both qualitatively and quantitatively, resulting in superior fused images with improved human perception quality.
引用
收藏
页码:237 / 248
页数:12
相关论文
共 50 条
  • [31] Multi-modal cardiac image fusion and visualization on the GPU
    Kiss, Gabriel
    Asen, Jon Petter
    Bogaert, Jan
    Amundsen, Brage
    Claus, Piet
    D'hooge, Jan
    Torp, Hans G.
    2011 IEEE INTERNATIONAL ULTRASONICS SYMPOSIUM (IUS), 2011, : 254 - 257
  • [32] Attention-based multi-modal fusion sarcasm detection
    Liu, Jing
    Tian, Shengwei
    Yu, Long
    Long, Jun
    Zhou, Tiejun
    Wang, Bo
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2023, 44 (02) : 2097 - 2108
  • [33] Multi-modal Perception Fusion Method Based on Cross Attention
    Zhang B.-L.
    Pan Z.-H.
    Jiang J.-Z.
    Zhang C.-B.
    Wang Y.-X.
    Yang C.-L.
    Zhongguo Gonglu Xuebao/China Journal of Highway and Transport, 2024, 37 (03): : 181 - 193
  • [34] Application of Multi-modal Fusion Attention Mechanism in Semantic Segmentation
    Liu, Yunlong
    Yoshie, Osamu
    Watanabe, Hiroshi
    COMPUTER VISION - ACCV 2022, PT VII, 2023, 13847 : 378 - 397
  • [35] TaGAT: Topology-Aware Graph Attention Network for Multi-modal Retinal Image Fusion
    Tian, Xin
    Anantrasirichai, Nantheera
    Nicholson, Lindsay
    Achim, Alin
    MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION - MICCAI 2024, PT I, 2024, 15001 : 775 - 784
  • [36] RCAFusion: Cross Rubik Cube Attention Network for Multi-modal Image Fusion of Intelligent Vehicles
    Li, Ang
    Yin, Guodong
    Wang, Ziwei
    Liang, Jinhao
    Wang, Fanxun
    Bai, Xin
    Liu, Zhichao
    2024 35TH IEEE INTELLIGENT VEHICLES SYMPOSIUM, IEEE IV 2024, 2024, : 2848 - 2854
  • [37] Multi-modal Conditional Attention Fusion for Dimensional Emotion Prediction
    Chen, Shizhe
    Jin, Qin
    MM'16: PROCEEDINGS OF THE 2016 ACM MULTIMEDIA CONFERENCE, 2016, : 571 - 575
  • [38] MULTI-MODAL HIERARCHICAL ATTENTION-BASED DENSE VIDEO CAPTIONING
    Munusamy, Hemalatha
    Sekhar, Chandra C.
    2023 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP, 2023, : 475 - 479
  • [39] Self-supervised multi-modal fusion network for multi-modal thyroid ultrasound image diagnosis
    Xiang, Zhuo
    Zhuo, Qiuluan
    Zhao, Cheng
    Deng, Xiaofei
    Zhu, Ting
    Wang, Tianfu
    Jiang, Wei
    Lei, Baiying
    COMPUTERS IN BIOLOGY AND MEDICINE, 2022, 150
  • [40] Adversarial Learning With Multi-Modal Attention for Visual Question Answering
    Liu, Yun
    Zhang, Xiaoming
    Huang, Feiran
    Cheng, Lei
    Li, Zhoujun
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2021, 32 (09) : 3894 - 3908