Multi-scale semantic image inpainting with residual learning and GAN

Image inpainting aims to fill the corrupt area semantically and recover the semantic and detailed information; however, concurrent methods suffer from convergence crash and arbitrarily paste the inference of the missing area into the corrupt context. In this paper, we study a combination of an encoder-decoder generator for image semantic inpainting and a multi-layer convolutional net for image seamless fusion, which is capable of restoring image effectively and seamlessly. Specifically, the encoder-decoder generator learns and extracts the latent compressed representations of missing areas from the context of a corrupt image, and further predicts a semantically correct estimation of the missing area based on the latent representations. The consecutive convolutional net smooths the discrepancy between the original image context and the estimation and seamlessly merges predictions and original images. The skip connections between the encoder and the decoder bridge the backward propagation of gradients, therefore boost the learning ability of the generator and stabilize the convergence of reconstruction loss. The performance and superiority of our method are illustrated and demonstrated on the real-world dataset qualitatively and quantitatively, and the experiments manifest acceptable semantic inpainting results, which significantly illustrates the effectiveness of our model. (C) 2018 Elsevier B.V. All rights reserved.