DGFusion: An effective dynamic generalizable network for infrared and visible image fusion

被引:0
|
作者
Zhao, Longjie [1 ]
Yang, Runlin [1 ]
Yan, Bao [1 ]
Wang, Song [2 ]
Guo, Zhengyu [3 ]
Luo, Delin [1 ]
机构
[1] Xiamen Univ, Sch Aerosp Engn, Xiamen 361102, Peoples R China
[2] China Southern Power Grid, Elect Power Res Inst, Guangzhou 510063, Peoples R China
[3] Natl Key Lab Air Based Informat Percept & Fus, Luoyang 471000, Peoples R China
关键词
Image fusion; Attention mechanism; Deep learning; Multi-modal; GENERATIVE ADVERSARIAL NETWORK; NEST;
D O I
10.1016/j.infrared.2024.105495
中图分类号
TH7 [仪器、仪表];
学科分类号
0804 ; 080401 ; 081102 ;
摘要
The objective of infrared and visible image fusion is to generate a unified image that highlights prominent targets and retains intricate texture details, even in scenarios with imbalanced source image information. However, current image fusion algorithms primarily consider factors like illumination, restricting their applicability to certain scenes and compromising their adaptability. To tackle the issue, this paper proposes the DGFusion, which utilizes TWSSLoss to balance the contribution of source images in the fused output, effectively mitigating the limitations linked to relying solely on illumination guidance. Additionally, modality- complement feature attention harmonizer (MCFAH) facilitates cross-modal channel attention learning. This process assigns weights to features and accomplishes fusion by exchanging cross-modal differential information, thereby enriching each feature with details from the other modality. Furthermore, the multi convolution attentive net (MCAN) dynamically adjusts the contributions of features from different modalities. It prioritizes the most expressive characteristics to accentuate complementary information, enabling efficient fusion. In conclusion, our method outperforms seven state-of-the-art alternatives in terms of preserving target details and retaining texture information. Rigorous generalization experiments across five diverse datasets demonstrate the robustness of our DGFusion model in various scenarios.
引用
收藏
页数:16
相关论文
共 50 条
  • [31] Self-Attention Progressive Network for Infrared and Visible Image Fusion
    Li, Shuying
    Han, Muyi
    Qin, Yuemei
    Li, Qiang
    [J]. REMOTE SENSING, 2024, 16 (18)
  • [32] Fully convolutional network-based infrared and visible image fusion
    Feng, Yufang
    Lu, Houqing
    Bai, Jingbo
    Cao, Lin
    Yin, Hong
    [J]. MULTIMEDIA TOOLS AND APPLICATIONS, 2020, 79 (21-22) : 15001 - 15014
  • [33] An Information Retention and Feature Transmission Network for Infrared and Visible Image Fusion
    Liu, Chang
    Yang, Bin
    Li, Yuehua
    Zhang, Xiaozhi
    Pang, Lihui
    [J]. IEEE SENSORS JOURNAL, 2021, 21 (13) : 14950 - 14959
  • [34] Laplacian Pyramid Generative Adversarial Network for Infrared and Visible Image Fusion
    Yin, Haitao
    Xiao, Jinghu
    [J]. IEEE SIGNAL PROCESSING LETTERS, 2022, 29 : 1988 - 1992
  • [35] VISIBLE AND INFRARED IMAGE FUSION USING ENCODER-DECODER NETWORK
    Ataman, Ferhat Can
    Bozdagi Akar, Gozde
    [J]. 2021 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2021, : 1779 - 1783
  • [36] RXDNFuse: A aggregated residual dense network for infrared and visible image fusion
    Long, Yongzhi
    Jia, Haitao
    Zhong, Yida
    Jiang, Yadong
    Jia, Yuming
    [J]. INFORMATION FUSION, 2021, 69 : 128 - 141
  • [37] RITFusion: Reinforced Interactive Transformer Network for Infrared and Visible Image Fusion
    Li, Xiaoling
    Li, Yanfeng
    Chen, Houjin
    Peng, Yahui
    Chen, Luyifu
    Wang, Minjun
    [J]. IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2024, 73
  • [38] Infrared and Visible Image Fusion Based on Significant Matrix and Neural Network
    Shen Yu
    Chen Xiaopeng
    Yuan Yubin
    Wang Lin
    Zhang Hongguo
    [J]. LASER & OPTOELECTRONICS PROGRESS, 2020, 57 (20)
  • [39] Frequency Integration and Spatial Compensation Network for infrared and visible image fusion
    Zheng, Naishan
    Zhou, Man
    Huang, Jie
    Zhao, Feng
    [J]. INFORMATION FUSION, 2024, 109
  • [40] FPNFuse: A lightweight feature pyramid network for infrared and visible image fusion
    Zhang, Zi-Han
    Wu, Xiao-Jun
    Xu, Tianyang
    [J]. IET IMAGE PROCESSING, 2022, 16 (09) : 2308 - 2320