Robust Dynamic Radiance Fields

被引:34
|
作者
Liu, Yu-Lun [2 ]
Gao, Chen [1 ]
Meuleman, Andreas [3 ]
Tseng, Hung-Yu [1 ]
Saraf, Ayush [1 ]
Kim, Changil [1 ]
Chuang, Yung-Yu [2 ]
Kopf, Johannes [1 ]
Huang, Jia-Bin [1 ,4 ]
机构
[1] Meta, Menlo Pk, CA 94025 USA
[2] Natl Taiwan Univ, Taipei, Taiwan
[3] Korea Adv Inst Sci & Technol, Daejeon, South Korea
[4] Univ Maryland, College Pk, MD USA
关键词
VIDEO;
D O I
10.1109/CVPR52729.2023.00010
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Dynamic radiance field reconstruction methods aim to model the time-varying structure and appearance of a dynamic scene. Existing methods, however, assume that accurate camera poses can be reliably estimated by Structure from Motion (SfM) algorithms. These methods, thus, are unreliable as SfM algorithms often fail or produce erroneous poses on challenging videos with highly dynamic objects, poorly textured surfaces, and rotating camera motion. We address this robustness issue by jointly estimating the static and dynamic radiance fields along with the camera parameters (poses and focal length). We demonstrate the robustness of our approach via extensive quantitative and qualitative experiments. Our results show favorable performance over the state-of-the-art dynamic view synthesis methods.
引用
收藏
页码:13 / 23
页数:11
相关论文
共 50 条
  • [1] Regularizing Dynamic Radiance Fields with Kinematic Fields
    Im, Woobin
    Cha, Geonho
    Lee, Sebin
    Lee, Jumin
    Seon, Juhyeong
    Wee, Dongyoon
    Yoon, Sung-Eui
    COMPUTER VISION - ECCV 2024, PT XXXIX, 2025, 15097 : 312 - 328
  • [2] Disentangled Generation and Aggregation for Robust Radiance Fields
    Shen, Shihe
    Gao, Huachen
    Xu, Wangze
    Peng, Rui
    Tang, Luyang
    Xiong, Kaiqiang
    Jiao, Jianbo
    Wang, Ronggang
    COMPUTER VISION - ECCV 2024, PT XLIX, 2025, 15107 : 218 - 236
  • [3] DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields
    Schischka, Nicolas
    Schieber, Hannah
    Karaoglu, Mert Asim
    Gorgulu, Melih
    Groetzner, Florian
    Ladikos, Alexander
    Navab, Nassir
    Roth, Daniel
    Busam, Benjamin
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2025, 10 (01): : 548 - 555
  • [4] WateRF: Robust Watermarks in Radiance Fields for Protection of Copyrights
    Jang, Youngdong
    Lee, Dong In
    Jang, MinHyuk
    Ki, Jong Wook
    Yang, Feng
    Kim, Sangpil
    2024 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2024, : 12087 - 12097
  • [5] Dynamic Mesh-Aware Radiance Fields
    Qiao, Yi-Ling
    Gao, Alexander
    Xu, Yiran
    Feng, Yue
    Huang, Jia-Bin
    Lin, Ming C.
    2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION, ICCV, 2023, : 385 - 396
  • [6] MuNeRF: Robust Makeup Transfer in Neural Radiance Fields
    Yuan, Yu-Jie
    Han, Xinyang
    He, Yue
    Zhang, Fang-Lue
    Gao, Lin
    IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2025, 31 (03) : 1746 - 1757
  • [7] Controllable Radiance Fields for Dynamic Face Synthesis
    Zhuang, Peiye
    Ma, Liqian
    Koyejo, Sanmi
    Schwing, Alexander
    2022 INTERNATIONAL CONFERENCE ON 3D VISION, 3DV, 2022, : 646 - 656
  • [8] Fast High Dynamic Range Radiance Fields for Dynamic Scenes
    Wu, Guanjun
    Yi, Taoran
    Fang, Jiemin
    Liu, Wenyu
    Wang, Xinggang
    2024 INTERNATIONAL CONFERENCE IN 3D VISION, 3DV 2024, 2024, : 862 - 872
  • [9] RoDyn-SLAM: Robust Dynamic Dense RGB-D SLAM With Neural Radiance Fields
    Jiang, Haochen
    Xu, Yueming
    Li, Kejie
    Feng, Jianfeng
    Zhang, Li
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2024, 9 (09): : 7509 - 7516
  • [10] Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation
    Lin, Yunzhi
    Mueller, Thomas
    Tremblay, Jonathan
    Wen, Bowen
    Tyree, Stephen
    Evans, Alex
    Vela, Patricio A.
    Birchfield, Stan
    2023 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA 2023), 2023, : 9377 - 9384