Remote Sensing Image Fusion Based on Generative Adversarial Network with Multi-stream Fusion Architecture

The generative adversarial network receives extensive attention in the study of computer vision such as image fusion and image super-resolution, due to its strong ability of generating high quality images. At present, the remote sensing image fusion method based on generative adversarial network only learns the mapping between the images, and lacks the unique Pan-sharpening domain knowledge. This paper proposes a remote sensing image fusion method based on optimized generative adversarial network with the integration of the spatial structure information of panchromatic image. The proposed algorithm extracts the spatial structure information of the panchromatic image by the gradient operator. The extracted feature would be added to both the discriminator and the generator which uses a multi-stream fusion architecture. The corresponding optimization objective and fusion rules are then designed to improve the quality of the fused image. Experiments on images acquired by WorldView-3 satellites demonstrate that the proposed method can generate high quality fused images, which is better than the most of advanced remote sensing image fusion methods in both subjective visual and objective evaluation indicators.