SFCFusion: Spatial-Frequency Collaborative Infrared and Visible Image Fusion

Infrared images can provide prominent targets based on the radiation difference, making them suitable for use in all day and night conditions. On the other hand, visible images can offer texture details with high spatial resolution. Infrared and visible image fusion is promising to achieve the best of both. Conventional frequency or spatial multiscale transformation (MST) methods are good at preserving image details. Deep-learning-based methods become more and more popular in image fusion because they can preserve high-level semantic features. To tackle the challenge in extracting and fusing cross-modality and cross-domain information, we propose a spatial-frequency collaborative fusion (SFCFusion) framework that effectively fuses spatial and frequency information in the feature space. In the frequency domain, source images are decomposed into base and detail layers with existing frequency decomposition methods. In the spatial domain, a kernel-based saliency generation module is designed to preserve spatial region-level structural information. A deep-learning-based encoder is used to extract features from the source images, decomposed images, and saliency maps. In the shared feature space, we achieve cross-modality SFCFusion through our proposed adaptive fusion scheme. We have conducted experiments to compare our SFCFusion with both the conventional and deep learning approaches on the TNO, LLVIP, and M3FD datasets. The qualitative and quantitative evaluation results demonstrate the effectiveness of our SFCFusion. We have further demonstrated the superiority of our SFCFusion in the downstream detection task. Our code will be available at https://github.com/ChenHanrui430/SFCFusion.