On the generalization of learning-based 3D reconstruction

被引:14
|
作者
Bautista, Miguel Angel [1 ]
Talbott, Walter [1 ]
Zhai, Shuangfei [1 ]
Srivastava, Nitish [1 ]
Susskind, Joshua M. [1 ]
机构
[1] Apple, Cupertino, CA 95014 USA
关键词
D O I
10.1109/WACV48630.2021.00223
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
State-of-the-art learning-based monocular 3D reconstruction methods learn priors over object categories on the training set, and as a result struggle to achieve reasonable generalization to object categories unseen during training. In this paper we study the inductive biases encoded in the model architecture that impact the generalization of learning-based 3D reconstruction methods. We find that 3 inductive biases impact performance: the spatial extent of the encoder, the use of the underlying geometry of the scene to describe point features, and the mechanism to aggregate information from multiple views. Additionally, we propose mechanisms to enforce those inductive biases: a point representation that is aware of camera position, and a variance cost to aggregate information across views. Our model achieves state-of-the-art results on the standard ShapeNet 3D reconstruction benchmark in various settings.
引用
收藏
页码:2179 / 2188
页数:10
相关论文
共 50 条
  • [1] Deep learning-based 3D reconstruction: a survey
    Samavati, Taha
    Soryani, Mohsen
    [J]. ARTIFICIAL INTELLIGENCE REVIEW, 2023, 56 (09) : 9175 - 9219
  • [2] Deep learning-based 3D reconstruction: a survey
    Taha Samavati
    Mohsen Soryani
    [J]. Artificial Intelligence Review, 2023, 56 : 9175 - 9219
  • [3] Deep learning-based 3D reconstruction of scaffolds using a robot dog
    Kim, Juhyeon
    Chung, Duho
    Kim, Yohan
    Kim, Hyoungkwan
    [J]. AUTOMATION IN CONSTRUCTION, 2022, 134
  • [4] Learning-based 3D surface optimization from medical image reconstruction
    Wei, Mingqiang
    Wang, Jun
    Guo, Xianglin
    Wu, Huisi
    Xie, Haoran
    Wang, Fu Lee
    Qin, Jing
    [J]. OPTICS AND LASERS IN ENGINEERING, 2018, 103 : 110 - 118
  • [5] Deep learning-based 3D reconstruction from multiple images: A survey
    Wang, Chuhua
    Reza, Md Alimoor
    Vats, Vibhas
    Ju, Yingnan
    Thakurdesai, Nikhil
    Wang, Yuchen
    Crandall, David J.
    Jung, Soon-heung
    Seo, Jeongil
    [J]. NEUROCOMPUTING, 2024, 597
  • [6] An improved deep learning-based algorithm for 3D reconstruction of vacuum arcs
    Wang, Zhenxing
    Pan, Yangbo
    Zhang, Wei
    Li, Haomin
    Geng, Yingsan
    Wang, Jianhua
    Sun, Liqiong
    [J]. REVIEW OF SCIENTIFIC INSTRUMENTS, 2021, 92 (12):
  • [7] LEARNING-BASED FULLY 3D FACE RECONSTRUCTION FROM A SINGLE IMAGE
    Hu, Xiaoping
    Wang, Ying
    Zhu, Feiyun
    Pan, Chunhong
    [J]. 2016 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING PROCEEDINGS, 2016, : 1651 - 1655
  • [9] Pavement crack measurement based on aerial 3D reconstruction and learning-based segmentation method
    Jiang, Shang
    Gu, Siyang
    Yan, Zhiyu
    [J]. MEASUREMENT SCIENCE AND TECHNOLOGY, 2023, 34 (01)
  • [10] Diagnostic performance of deep learning-based reconstruction algorithm in 3D MR neurography
    Ensle, Falko
    Kaniewska, Malwina
    Tiessen, Anja
    Lohezic, Maelene
    Getzmann, Jonas M.
    Guggenberger, Roman
    [J]. SKELETAL RADIOLOGY, 2023, 52 (12) : 2409 - 2418