HFHFusion: A Heterogeneous Feature Highlighted method for infrared and visible image fusion

Recently, infrared and visible image fusion has attracted considerable attention from researchers. Under extreme or low resolution conditions, the existing image fusion algorithms are easily misled by redundant information in visible images. It is difficult to maintain thermal radiation targets in infrared images clearly. To address this issue, we introduce an AutoEncoder framework for image fusion named HFHFusion(heterogeneous feature highlighted Fusion Network), which integrates a heterogeneous feature extraction network and an RCF edge detection network.The HFHFusion framework introduces a heterogeneous two-branch feature extraction structure, leveraging distinct feature extraction methods tailored to diverse sensor inputs. Firstly, we design an edge extraction network specialized for visible images, aimed at capturing detailed texture information effectively. Secondly, we devise an infrared image contrast enhancement network leveraging a channel attention mechanism, directly linked to an encoder. Our experiments demonstrate that this network facilitates better integration of infrared thermal radiation information into the fusion results. To effectively integrate image features from multimodal scenes into a unified network, we propose a heterogeneous feature extraction network, combining an AutoEncoder structure with a CNN structure. It underscores the significance of the feature extraction process in image fusion tasks. Extensive experiments conducted on public datasets highlight the advantages of our HFHFusion over state-of-the-art image fusion algorithms and task-specific image fusion methods. HFHFusion was subjected to corresponding fusion test experiments on the TNO, RoadScene and M3FD data sets respectively and compared with 8 excellent fusion algorithms. The experiments have demonstrated that our method is more suitable for extreme conditions with complex environments and low resolutions.