Self-supervised reflectance-guided 3d shape reconstruction from single-view images

3D shape reconstruction from a single-view image is an utterly ill-posed and challenging problem, while multi-view methods can reconstruct an object’s shape only from raw images. However, these raw images should be shot in a static scene, to promise that corresponding features in the images can be mapped to the same spatial location. Recent single-view methods need only single-view images of static or dynamic objects, by turning to prior knowledge to mine the latent multi-view information in single-view images. Some of them utilize prior models (e.g. rendering-based or style-transfer-based) to generate novel-view images, which are however not sufficiently accurate, to feed their model. In this paper, we represent Augmented Self-Supervised 3D Reconstruction with Monotonous Material (ASRMM) approach, trained end-to-end in a self-supervised manner, to obtain the 3D reconstruction of a category-specific object, without any relevant prior models for novel-view images. Our approach draws inspiration from the experience that (1) high quality multi-view images are difficult to obtain, and (2) the shape of an object of single material can be visually inferred more easily, rather than of multiple kinds of complex material. As to practice these motivations, ASRMM makes material monotonous in its diffuse part by setting reflectance an identical value, and apply this idea on the source and reconstruction images. Experiments show that our model can reasonably reconstruct the 3D model of faces, cats, cars and birds from their collections of single-view images, and the experiments also show that our approach can be generalized to different reconstruction tasks, including unsupervised depth-based reconstruction and 2D supervised mesh reconstruction, and achieve promising improvement in the quality of the reconstructed shape and the texture.