CNN-Based Stereoscopic Image Inpainting

CNN has proved powerful in many tasks, including single image inpainting. The paper presents an end-to-end network for stereoscopic image inpainting. The proposed network is composed of two encoders for independent feature extraction of a pair of stereo images with missing regions, a feature fusion module for stereo coherent structure prediction, and two decoders to generate a pair of completed images. In order to train the model, besides a reconstruction and an adversarial loss for content recovery, a local consistency loss is defined to constrain stereo coherent detail prediction. Moreover, we present a transfer-learning based training strategy to solve the issue of stereoscopic data scarcity. To the best of our knowledge, we are the first to solve the stereoscopic inpainting problem in the framework of CNN. Compared to traditional stereoscopic inpainting and available CNN-based single image inpainting (repairing stereo views one by one) methods, our network generates results of higher image quality and stereo consistency.