Robust Dynamic Radiance Fields

被引:34
|
作者
Liu, Yu-Lun [2 ]
Gao, Chen [1 ]
Meuleman, Andreas [3 ]
Tseng, Hung-Yu [1 ]
Saraf, Ayush [1 ]
Kim, Changil [1 ]
Chuang, Yung-Yu [2 ]
Kopf, Johannes [1 ]
Huang, Jia-Bin [1 ,4 ]
机构
[1] Meta, Menlo Pk, CA 94025 USA
[2] Natl Taiwan Univ, Taipei, Taiwan
[3] Korea Adv Inst Sci & Technol, Daejeon, South Korea
[4] Univ Maryland, College Pk, MD USA
关键词
VIDEO;
D O I
10.1109/CVPR52729.2023.00010
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Dynamic radiance field reconstruction methods aim to model the time-varying structure and appearance of a dynamic scene. Existing methods, however, assume that accurate camera poses can be reliably estimated by Structure from Motion (SfM) algorithms. These methods, thus, are unreliable as SfM algorithms often fail or produce erroneous poses on challenging videos with highly dynamic objects, poorly textured surfaces, and rotating camera motion. We address this robustness issue by jointly estimating the static and dynamic radiance fields along with the camera parameters (poses and focal length). We demonstrate the robustness of our approach via extensive quantitative and qualitative experiments. Our results show favorable performance over the state-of-the-art dynamic view synthesis methods.
引用
收藏
页码:13 / 23
页数:11
相关论文
共 50 条
  • [41] TensoRF: Tensorial Radiance Fields
    Chen, Anpei
    Xu, Zexiang
    Geiger, Andreas
    Yu, Jingyi
    Su, Hao
    COMPUTER VISION - ECCV 2022, PT XXXII, 2022, 13692 : 333 - 350
  • [42] Generalizing Neural Radiance Fields for Robust 6D Pose Estimation of Unseen Appearances
    Poellabauer, Thomas
    Wirth, Tristan
    Weitz, Paul
    Knauthe, Volker
    Kuijper, Arjan
    Fellner, Dieter W.
    ADVANCES IN VISUAL COMPUTING, ISVC 2024, PT I, 2025, 15046 : 300 - 314
  • [43] Learning Robust Multi-scale Representation for Neural Radiance Fields from Unposed Images
    Jain, Nishant
    Kumar, Suryansh
    Van Gool, Luc
    INTERNATIONAL JOURNAL OF COMPUTER VISION, 2024, 132 (04) : 1310 - 1335
  • [44] Learning Robust Multi-scale Representation for Neural Radiance Fields from Unposed Images
    Nishant Jain
    Suryansh Kumar
    Luc Van Gool
    International Journal of Computer Vision, 2024, 132 : 1310 - 1335
  • [45] Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis
    Shen, Shuai
    Li, Wanhua
    Zhu, Zheng
    Duan, Yueqi
    Zhou, Jie
    Lu, Jiwen
    COMPUTER VISION, ECCV 2022, PT XII, 2022, 13672 : 666 - 682
  • [46] S-DyRF: Reference-Based Stylized Radiance Fields for Dynamic Scenes
    Li, Xingyi
    Cao, Zhiguo
    Wu, Yizheng
    Wang, Kewei
    Xian, Ke
    Wang, Zhe
    Lin, Guosheng
    2024 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2024, : 20102 - 20112
  • [47] HDR-Plenoxels: Self-Calibrating High Dynamic Range Radiance Fields
    Jun-Seong, Kim
    Yu-Ji, Kim
    Ye-Bin, Moon
    Oh, Tae-Hyun
    COMPUTER VISION - ECCV 2022, PT XXXII, 2022, 13692 : 384 - 401
  • [48] Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modeling
    Kim, Jaehyeok
    Wee, Dongyoon
    Xu, Dan
    arXiv,
  • [49] VideoRF: Rendering Dynamic Radiance Fields as 2D Feature Video Streams
    Wang, Liao
    Yao, Kaixin
    Guo, Chengcheng
    Zhang, Zhirui
    Hu, Qiang
    Yu, Jingyi
    Xu, Lan
    Wu, Minye
    2024 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2024, 2024, : 470 - 481
  • [50] Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
    Gafni, Guy
    Thies, Justus
    Zollhoefer, Michael
    Niessner, Matthias
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 8645 - 8654