GIRAFFE HD: A High-Resolution 3D-aware Generative Model

被引:20
|
作者
Xue, Yang [1 ]
Li, Yuheng [2 ]
Singh, Krishna Kumar [3 ]
Lee, Yong Jae [2 ]
机构
[1] Univ Calif Davis, Davis, CA 95616 USA
[2] UW Madison, Madison, WI USA
[3] Adobe Res, San Jose, CA USA
关键词
D O I
10.1109/CVPR52688.2022.01789
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
3D-aware generative models have shown that the introduction of 3D information can lead to more controllable image generation. In particular, the current state-of-the-art model GIRAFFE I j can control each object's rotation, translation, scale, and scene camera pose without corresponding supervision. However, GIRAFFE only operates well when the image resolution is low We propose GIRAFFE HD, a high-resolution 3D-aware generative model that inherits all of GIRAFFE's controllable features while generating high -quality, high -resolution images (5122 resolution and above). The key idea is to leverage a style based neural renderer, and to independently generate the foreground and background to force their disentanglement while imposing consistency constraints to stitch them together to composite a coherent final image. We demonstrate state-of-the-art 3D controllable high -resolution image generation on multiple natural image datasets.
引用
收藏
页码:18419 / 18428
页数:10
相关论文
共 50 条
  • [1] 3D-aware Blending with Generative NeRFs
    Kim, Hyunsu
    Lee, Gayoung
    Choi, Yunjey
    Kim, Jin-Hwa
    Zhu, Jun-Yan
    [J]. 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 22849 - 22861
  • [2] VQ3D: Learning a 3D-Aware Generative Model on ImageNet
    Sargent, Kyle
    Koh, Jing Yu
    Zhang, Han
    Chang, Huiwen
    Herrmann, Charles
    Srinivasan, Pratul
    Wu, Jiajun
    Sun, Deqing
    [J]. 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION, ICCV, 2023, : 4217 - 4227
  • [3] IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-aware Portrait Synthesis
    Sun, Jingxiang
    Wang, Xuan
    Shi, Yichun
    Wang, Lizhen
    Wang, Jue
    Liu, Yebin
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2022, 41 (06):
  • [4] 3D-Aware Semantic-Guided Generative Model for Human Synthesis
    Zhang, Jichao
    Sangineto, Enver
    Tang, Hao
    Siarohin, Aliaksandr
    Zhong, Zhun
    Sebe, Nicu
    Wang, Wei
    [J]. COMPUTER VISION - ECCV 2022, PT XV, 2022, 13675 : 339 - 356
  • [5] 3D-Aware Generative Model for Improved Side-View Image Synthesis
    Jo, Kyungmin
    Jin, Wonjoon
    Choo, Jaegul
    Lee, Hyunjoon
    Cho, Sunghyun
    [J]. 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 22805 - 22815
  • [6] A Survey on Deep Generative 3D-aware Image Synthesis
    Xia, Weihao
    Xue, Jing-Hao
    [J]. ACM COMPUTING SURVEYS, 2024, 56 (04)
  • [7] Generative Multiplane Neural Radiance for 3D-Aware Image Generation
    Kumar, Amandeep
    Bhunia, Ankan Kumar
    Narayan, Sanath
    Cholakkal, Hisham
    Anwer, Rao Muhammad
    Khan, Salman
    Yang, Ming-Hsuan
    Khan, Fahad Shahbaz
    [J]. 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION, ICCV, 2023, : 7354 - 7364
  • [8] GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
    Schwarz, Katja
    Liao, Yiyi
    Niemeyer, Michael
    Geiger, Andreas
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
  • [9] GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation
    Deng, Yu
    Yang, Jiaolong
    Xiang, Jianfeng
    Tong, Xin
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2022, : 10663 - 10673
  • [10] Generative Novel View Synthesis with 3D-Aware Diffusion Models
    Chan, Eric R.
    Nagano, Koki
    Chan, Matthew A.
    Bergman, Alexander W.
    Park, Jeong Joon
    Levy, Axel
    Aittala, Miika
    De Mello, Shalini
    Karras, Tero
    Wetzstein, Gordon
    [J]. 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION, ICCV, 2023, : 4194 - 4206