Deep learning-based 3D reconstruction: a survey

Image-based 3D reconstruction is a long-established, ill-posed problem defined within the scope of computer vision and graphics. The purpose of image-based 3D reconstruction is to retrieve the 3D structure and geometry of a target object or scene from a set of input images. This task has a wide range of applications in various fields, such as robotics, virtual reality, and medical imaging. In recent years, learning-based methods for 3D reconstruction have attracted many researchers worldwide. These novel methods can implicitly estimate the 3D shape of an object or a scene in an end-to-end manner, eliminating the need for developing multiple stages such as key-point detection and matching. Furthermore, these novel methods can reconstruct the shapes of objects from a single input image. Due to rapid advancements in this field, as well as the multitude of opportunities to improve the performance of 3D reconstruction methods, a thorough review of algorithms in this area seems necessary. As a result, this research provides a complete overview of recent developments in the field of image-based 3D reconstruction. The studied methods are examined from several viewpoints, such as input types, model structures, output representations, and training strategies. A detailed comparison is also provided for the reader. Finally, unresolved challenges, underlying issues, and possible future work are discussed.