Generative Adversarial Network for Mural Inpainting Based on Multi-level Feature Fusion and Hypergraph Convolution

Aiming at the problems of low feature utilization and insufficient attention to context information in existing deep learning image restoration methods, a generative adversarial mural restoration model based on multi-level feature fusion and hypergraph convolution was proposed. Firstly, a pyramid feature extraction layer structure was designed, and the multi-scale feature extraction was carried out by using the pyramid feature layer. The mixed dilated convolution unit was used to expand the receptive field of multi-layer feature extraction, which overcomes the problem of insufficient feature extraction ability of single-scale convolution operation. Then, a multi-branch short chain fusion layer and a gating method were proposed to fuse the multi branch features, and the feature information between the adjacent branches was fused, so that the fused mural feature map has both the same branch features and the adjacent branch features, improving the utilization rate of feature information, and introducing the gating mechanism to select and fuse features to reduce the loss of detail information. Next, the fused features were passed through the ConvLSTM feature attention to enhance the attention to the mural context information. Finally, a hypergraph convolution mural long-range feature enhancement module was designed. By establishing a hypergraph convolution layer between the skip connection of the encoder and the decoder, the spatial feature information of the encoder was captured by hypergraph convolution and transferred to the decoder, which helpes the decoder to generate mural images and strengthenes the long-range dependence of features. Afterwards, the mural restoration was completed in a game against the SN–PatchGAN discriminator. Through the restoration experiments of digital Dunhuang murals, the results showed that the proposed method is superior to the comparative algorithms, and the restoration results of damaged murals are clearer and more natural. © 2024 Sichuan University. All rights reserved.