Unsupervised Facial Image De-occlusion with Optimized Deep Generative Models

In recent years, Generative Adversarial Networks (GANs) or various types of Auto-Encoders (AEs) have gained attention on facial image de-occlusion and/or in-painting tasks. In this paper, we propose a novel unsupervised technique to remove occlusion from facial images and complete the occluded parts simultaneously with optimized Deep Convolutional Generative Adversarial Networks (DCGANs) in an iterative way. Generally, GANs, as generative models, can estimate the distribution of images using a generator and a discriminator. DCGANs, as its variant, are proposed to conquer its instability during training. Existing facial image in-painting methods manually define a block of pixels as the missing part and the potential content of this block is semantically generated using generative models, such as GANs or AEs. In our method, a mask is inferred from an occluded facial image using a novel loss function, and then this mask is utilized to in-paint the occlusions automatically by pre-trained DCGANs. We evaluate the performance of our method on facial images with various occlusions, such as sunglasses and scarves. The experiments demonstrate that our method can effectively detect certain kinds of occlusions and complete the occluded parts in an unsupervised manner.