MIMFormer: Multiscale Inception Mixer Transformer for Hyperspectral and Multispectral Image Fusion

The fusion of low-spatial-resolution hyperspectral image and high-spatial-resolution multispectral image provides an effective method to obtain high-spatial-resolution hyperspectral image. However, existing hybrid fusion architectures combining convolutional neural networks (CNNs) and transformers face significant challenges. Sequential approaches struggle with simultaneous local and global modeling, while parallel approaches often result in information redundancy. In this article, to meet diverse information demands at different layers, we propose a novel multiscale inception mixer transformer network (MIMFormer), a multiscale hybrid network based on the Inception structure integrating CNN and transformer. The core of this network is the multiscale spatial transformer (MST) structure, which enhances the detail richness of fused images by integrating local and global information at various scales. The inception spatial-spectral mixer (ISSM) module within the MST leverages an Inception architecture and employs a spectral splitting mechanism to regulate spectral channel counts across different branches. This design allows the ISSM module to efficiently extract local spatial-spectral features through convolution and max pooling, while global features are captured using a self-attention mechanism, ensuring comprehensive feature fusion across spectral groups. Experimental results on three benchmark datasets and one real remote sensing dataset demonstrate that MIMFormer outperforms ten advanced fusion methods.