3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

3D object reconstruction has always been a hot topic in computer vision. Especially in recent years, many methods of learning volumetric predictions achieve robust 3D reconstruction using deep networks with 3D convolutional operations, which are direct analogies to classical 2D ones. However, the majority of methods employ strong shape priors and exist computationally waste in predicting 3D shapes. In this paper, we propose 3D-Mask-GAN, a novel framework to efficiently accomplish the task of unsupervised single-view 3D object reconstruction. We use 3D Generative Adversarial Networks (GAN) to predict the 3D shape from the single-view image and improve reconstruction accuracy by applying 2D projection masks instead of 3D priors simultaneously. The key idea is to insert a projector (a buildin camera system to approximate the true rendering pipeline) into the framework of 3D GAN to synthesize novel masks for optimization. We learn single-class and multi-class objects to evaluate our network. Experimental results show that our framework achieves impressive performance with fewer training iterations in terms of unsupervised shape predictions.