SMAE-Fusion: Integrating saliency-aware masked autoencoder with hybrid attention transformer for infrared-visible image fusion

The objective of infrared-visible image fusion (IVIF) is to generate composite images from multiple modalities that enhance visual representation and support advanced vision tasks. However, most existing IVIF methods primarily focus on enhancing visual effects, while high-level task-driven approaches are constrained by specific perception networks and complex training strategies, leading to limited flexibility across diverse scenarios. Emerging as a powerful self-supervised training paradigm, masked image modeling enables the learning of robust feature representations applicable to various downstream tasks. For this purpose, this study introduces SMAE-Fusion, a saliency-aware masked autoencoder framework tailored for infrared-visible image fusion. Initially, SMAE-Fusion adopts a saliency-aware dynamic masking strategy and utilizes the self-supervised pre-training paradigm in reconstruction stage to adaptively emphasize salient regions and semantic details, thereby improving feature representation to mitigate the semantic gap between upstream and downstream tasks. Moreover, the backbone of SMAE-Fusion incorporates the hybrid attention-enhanced transformer to promote effective interaction between local and global features by leveraging both convolutional and self- attention mechanisms. Additionally, a progressive feature fusion module is constructed to gradually optimize the integration of cross-modal features through self-attention alignment and cross-attention complementation. Comprehensive experiments on various public datasets demonstrate that SMAE-Fusion attains state-of-the-art performance in fusion quality and downstream task enhancement.