3D Generative Model Latent Disentanglement via Local Eigenprojection

被引:1
|
作者
Foti, Simone [1 ]
Koo, Bongjin [1 ,2 ]
Stoyanov, Danail [1 ]
Clarkson, Matthew J. [1 ]
机构
[1] UCL, London, England
[2] Univ Calif Santa Barbara, Santa Barbara, CA USA
基金
英国惠康基金;
关键词
disentanglement; generative adversarial networks; geometric deep learning; variational autoencoder;
D O I
10.1111/cgf.14793
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Designing realistic digital humans is extremely complex. Most data-driven generative models used to simplify the creation of their underlying geometric shape do not offer control over the generation of local shape attributes. In this paper, we overcome this limitation by introducing a novel loss function grounded in spectral geometry and applicable to different neural-network-based generative models of 3D head and body meshes. Encouraging the latent variables of mesh variational autoencoders (VAEs) or generative adversarial networks (GANs) to follow the local eigenprojections of identity attributes, we improve latent disentanglement and properly decouple the attribute creation. Experimental results show that our local eigenprojection disentangled (LED) models not only offer improved disentanglement with respect to the state-of-the-art, but also maintain good generation capabilities with training times comparable to the vanilla implementations of the models. Our code and pre-trained models are available at .
引用
收藏
页数:13
相关论文
共 50 条
  • [1] Global-to-Local Generative Model for 3D Shapes
    Wang, Hao
    Schor, Nadav
    Hu, Ruizhen
    Huang, Haibin
    Cohen-Or, Daniel
    Huang, Hui
    [J]. ACM TRANSACTIONS ON GRAPHICS, 2018, 37 (06):
  • [2] Global-to-Local Generative Model for 3D Shapes
    Wang, Hao
    Schor, Nadav
    Hu, Ruizhen
    Huang, Haibin
    Cohen-Or, Daniel
    Huang, Hui
    [J]. SIGGRAPH ASIA'18: SIGGRAPH ASIA 2018 TECHNICAL PAPERS, 2018,
  • [3] Learning Distribution Independent Latent Representation for 3D Face Disentanglement
    Zhang, Zihui
    Yu, Cuican
    Li, Huibin
    Sun, Jian
    Liu, Feng
    [J]. 2020 INTERNATIONAL CONFERENCE ON 3D VISION (3DV 2020), 2020, : 848 - 857
  • [4] HiStyle: Reinventing historic portraits via 3D generative model
    Chen, Zhuo
    Yang, Rong
    Yan, Yichao
    Li, Zhu
    [J]. DISPLAYS, 2024, 83
  • [5] 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces
    Foti, Simone
    Koo, Bongjin
    Stoyanov, Danail
    Clarkson, Matthew J.
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 18709 - 18718
  • [6] NaviNeRF: NeRF-based 3D Representation Disentanglement by Latent Semantic Navigation
    Xie, Baao
    Li, Bohan
    Zhang, Zequn
    Dong, Junting
    Jin, Xin
    Yang, Jingyu
    Zeng, Wenjun
    [J]. 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 17946 - 17956
  • [7] Generating diverse clothed 3D human animations via a generative model
    Shi, Min
    Feng, Wenke
    Gao, Lin
    Zhu, Dengming
    [J]. COMPUTATIONAL VISUAL MEDIA, 2024, 10 (02) : 261 - 277
  • [8] Generating diverse clothed 3D human animations via a generative model
    Min Shi
    Wenke Feng
    Lin Gao
    Dengming Zhu
    [J]. Computational Visual Media, 2024, 10 : 261 - 277
  • [9] Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
    Wu, Jiajun
    Zhang, Chengkai
    Xue, Tianfan
    Freeman, William T.
    Tenenbaum, Joshua B.
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 29 (NIPS 2016), 2016, 29
  • [10] Dynamical Deep Generative Latent Modeling of 3D Skeletal Motion
    Amirreza Farnoosh
    Sarah Ostadabbas
    [J]. International Journal of Computer Vision, 2022, 130 : 2695 - 2706