Viewpoint-independent Single-view 3D Object Reconstruction using Reinforcement Learning

This paper addresses the problem of reconstructing 3D object shapes from single-view images using reinforcement learning. Reinforcement learning allows us to interpret the reconstruction process of a 3D object by visualizing sequentially selected actions. However, the conventional method used a single fixed viewpoint and was not validated with an arbitrary viewpoint. To handle images from arbitrary viewpoints, we propose a reinforcement learning framework that introduces an encoder to extract viewpoint-independent image features. We train an encoder-decoder network to disentangle shape and viewpoint features from the image. The parameters of the encoder part of the network are fixed, and the encoder is incorporated into the reinforcement learning framework as an image feature extractor. Since the encoder learns to extract viewpoint-independent features from images of arbitrary viewpoints, only images of a single viewpoint are needed for reinforcement learning. The experimental results show that the proposed method can learn faster and achieves better accuracy than the conventional method.