CCAFusion: Cross-Modal Coordinate Attention Network for Infrared and Visible Image Fusion

被引:2
|
作者
Li, Xiaoling [1 ]
Li, Yanfeng [1 ]
Chen, Houjin [1 ]
Peng, Yahui [1 ]
Pan, Pan [1 ]
机构
[1] Beijing Jiaotong Univ, Sch Elect & Informat Engn, Beijing 100044, Peoples R China
关键词
Image fusion; Feature extraction; Task analysis; Transforms; Generative adversarial networks; Decoding; Dictionaries; Infrared and visible image fusion; attention mechanism; cross-modal fusion strategy; coordinate attention; multiple constrained loss function; MULTI-FOCUS; PERFORMANCE; EFFICIENT; NEST;
D O I
10.1109/TCSVT.2023.3293228
中图分类号
TM [电工技术]; TN [电子技术、通信技术];
学科分类号
0808 ; 0809 ;
摘要
Infrared and visible image fusion aims to generate one image with comprehensive information. It can maintain rich texture characteristics and thermal information. However, for existing image fusion methods, the fused images either sacrifice the salience of thermal targets and the richness of textures or introduce the interference of useless information like artifacts. To alleviate these problems, an effective cross-modal coordinate attention network for infrared and visible image fusion called CCAFusion is proposed in this paper. To fully integrate complementary features, the cross-modal image fusion strategy based on coordinate attention is designed, which consists of the feature-awareness fusion module and the feature-enhancement fusion module. Moreover, a multiscale skip connection-based network is employed to obtain multiscale features in the infrared image and the visible image, which can fully utilize the multi-level information in the fusion process. To reduce the discrepancy between the fused image and the input images, a multiple constrained loss function including the base loss and the auxiliary loss is developed to adjust the gray-level distribution and ensure the harmonious coexistence of structure and intensity in fused images, thereby preventing the pollution of useless information like artifacts. Extensive experiments conducted on widely used datasets demonstrate that our CCAFusion achieves superior performance over state-of-the-art image fusion methods in both qualitative evaluation and quantitative measurement. Furthermore, the application to salient object detection reveals the potential of our CCAFusion for high-level vision tasks, which can effectively boost the detection performance.
引用
收藏
页码:866 / 881
页数:16
相关论文
共 50 条
  • [1] Cross-Modal Transformers for Infrared and Visible Image Fusion
    Park, Seonghyun
    Vien, An Gia
    Lee, Chul
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2024, 34 (02) : 770 - 785
  • [2] Infrared and visible image fusion based on cross-modal extraction strategy
    Liu, Xiaowen
    Li, Jing
    Yang, Xin
    Huo, Hongtao
    [J]. INFRARED PHYSICS & TECHNOLOGY, 2022, 124
  • [3] Efficient multi-level cross-modal fusion and detection network for infrared and visible image
    Gao, Hongwei
    Wang, Yutong
    Sun, Jian
    Jiang, Yueqiu
    Gai, Yonggang
    Yu, Jiahui
    [J]. ALEXANDRIA ENGINEERING JOURNAL, 2024, 108 : 306 - 318
  • [4] CMFA_Net: A cross-modal feature aggregation network for infrared-visible image fusion
    Ding, Zhaisheng
    Li, Haiyan
    Zhou, Dongming
    Li, Hongsong
    Liu, Yanyu
    Hou, Ruichao
    [J]. INFRARED PHYSICS & TECHNOLOGY, 2021, 118
  • [5] BCMFIFuse: A Bilateral Cross-Modal Feature Interaction-Based Network for Infrared and Visible Image Fusion
    Gao, Xueyan
    Liu, Shiguang
    [J]. REMOTE SENSING, 2024, 16 (17)
  • [6] MCFusion: infrared and visible image fusion based multiscale receptive field and cross-modal enhanced attention mechanism
    Jiang, Min
    Wang, Zhiyuan
    Kong, Jun
    Zhuang, Danfeng
    [J]. JOURNAL OF ELECTRONIC IMAGING, 2024, 33 (01)
  • [7] Cross-modal image fusion guided by subjective visual attention
    Fang, Aiqing
    Zhao, Xinbo
    Zhang, Yanning
    [J]. NEUROCOMPUTING, 2020, 414 (414) : 333 - 345
  • [8] CMFuse: Cross-Modal Features Mixing via Convolution and MLP for Infrared and Visible Image Fusion
    Cai, Zhao
    Ma, Yong
    Huang, Jun
    Mei, Xiaoguang
    Fan, Fan
    Zhao, Zhiqing
    [J]. IEEE SENSORS JOURNAL, 2024, 24 (15) : 24152 - 24167
  • [9] Multigrained Attention Network for Infrared and Visible Image Fusion
    Li, Jing
    Huo, Hongtao
    Li, Chang
    Wang, Renhua
    Sui, Chenhong
    Liu, Zhao
    [J]. IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, 2021, 70
  • [10] CEFusion: An Infrared and Visible Image Fusion Network Based on Cross-Modal Multi-Granularity Information Interaction and Edge Guidance
    Yang, Bin
    Hu, Yuxuan
    Liu, Xiaowen
    Li, Jing
    [J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2024,