3D structure inference by integrating segmentation and reconstruction from a single image

The authors present a hierarchical Bayesian method for inferring the 3D structure of polyhedral man-made objects from a single image by integrating 2D image parsing and 3D reconstruction. In the first stage, the image is parsed into its constituent components - arbitrary shape regions and polygonal shape regions. In the second stage, polygonal shape regions are grouped into man-made polyhedral objects. The 3D structures of these polyhedral objects are further inferred using geometric priors. These two stages are integrated into a Bayesian inference scheme and cooperate to compute the optimal solutions. This method enables the model to correct possible errors and explain ambiguities in the lower level with the help of information from the higher level. The algorithm is applied to the images of indoor scenes, and the experimental results demonstrate satisfactory performance.