DUFuse: Deep U-Net for visual and infrared images fusion

The vision sensor is capable of capturing image detail features suitable for human observation, while the infrared sensor is capable of capturing the thermal characteristics of the target object. Therefore, the vision and infrared image fusion aim to retain both the rich detail features in the visual image and the thermal characteristics of the infrared image. This study proposes a novel deep U-Net network model to solve the fusion task. First, an improved deep model is proposed for better feature extraction by borrowing the process of decomposition, fusion and reconstruction in the multiscale decomposition process. Second, structural similarity is introduced into the loss function, which enables the network to enhance the quality of the detailed features of the generated images. Third, we propose a new hierarchical fusion strategy as well as average fusion and weighted fusion rules. Extensive experiments demonstrate that the proposed algorithm is superior to state-of-the-art algorithms. © 2022, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.