Image-to-Image Translation With Disentangled Latent Vectors for Face Editing

被引:2
|
作者
Dalva, Yusuf [1 ]
Pehlivan, Hamza [2 ]
Hatipoglu, Oyku Irmak [3 ]
Moran, Cansu [4 ]
Dundar, Aysegul [5 ]
机构
[1] Virginia Tech, Dept Comp Sci, Blacksburg, VA 24061 USA
[2] Max Planck Inst, D-80539 Munich, Germany
[3] Swiss Fed Inst Technol, CH-8092 Zurich, Switzerland
[4] Tech Univ Munich, D-80333 Munich, Germany
[5] Bilkent Univ, Dept Comp Sci, TR-06800 Bilkent, Turkiye
关键词
Image translation; generative adversarial net works; latent space manipulation; face attribute editing;
D O I
10.1109/TPAMI.2023.3308102
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We propose an image-to-image translation framework for facial attribute editing with disentangled interpretable latent directions. Facial attribute editing task faces the challenges of targeted attribute editing with controllable strength and disentanglement in the representations of attributes to preserve the other attributes during edits. For this goal, inspired by the latent space factorization works of fixed pretrained GANs, we design the attribute editing by latent space factorization, and for each attribute, we learn a linear direction that is orthogonal to the others. We train these directions with orthogonality constraints and disentanglement losses. To project images to semantically organized latent spaces, we set an encoder-decoder architecture with attention-based skip connections. We extensively compare with previous image translation algorithms and editing with pretrained GAN works. Our extensive experiments show that our method significantly improves over the state-of-the-arts.
引用
收藏
页码:14777 / 14788
页数:12
相关论文
共 50 条
  • [1] Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation
    Liu, Yahui
    Sangineto, Enver
    Chen, Yajing
    Bao, Linchao
    Zhang, Haoxian
    Sebe, Nicu
    Lepri, Bruno
    Wang, Wei
    De Nadai, Marco
    [J]. 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 10780 - 10789
  • [2] Diverse Image-to-Image Translation via Disentangled Representations
    Lee, Hsin-Ying
    Tseng, Hung-Yu
    Huang, Jia-Bin
    Singh, Maneesh
    Yang, Ming-Hsuan
    [J]. COMPUTER VISION - ECCV 2018, PT I, 2018, 11205 : 36 - 52
  • [3] Style-Guided and Disentangled Representation for Robust Image-to-Image Translation
    Choi, Jaewoong
    Kim, Daeha
    Song, Byung Cheol
    [J]. THIRTY-SIXTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FOURTH CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE / THE TWELVETH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2022, : 463 - 471
  • [4] DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
    Hsin-Ying Lee
    Hung-Yu Tseng
    Qi Mao
    Jia-Bin Huang
    Yu-Ding Lu
    Maneesh Singh
    Ming-Hsuan Yang
    [J]. International Journal of Computer Vision, 2020, 128 : 2402 - 2417
  • [5] VecGAN: Image-to-Image Translation with Interpretable Latent Directions
    Dalva, Yusuf
    Altindis, Said Fahri
    Dundar, Aysegul
    [J]. COMPUTER VISION - ECCV 2022, PT XVI, 2022, 13676 : 153 - 169
  • [6] DRIT plus plus : Diverse Image-to-Image Translation via Disentangled Representations
    Lee, Hsin-Ying
    Tseng, Hung-Yu
    Mao, Qi
    Huang, Jia-Bin
    Lu, Yu-Ding
    Singh, Maneesh
    Yang, Ming-Hsuan
    [J]. INTERNATIONAL JOURNAL OF COMPUTER VISION, 2020, 128 (10-11) : 2402 - 2417
  • [7] Latent Filter Scaling for Multimodal Unsupervised Image-to-Image Translation
    Alharbi, Yazeed
    Smith, Neil
    Wonka, Peter
    [J]. 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 1458 - 1466
  • [8] Homomorphic Latent Space Interpolation for Unpaired Image-To-Image Translation
    Chen, Ying-Cong
    Xu, Xiaogang
    Tian, Zhuotao
    Jia, Jiaya
    [J]. 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 2403 - 2411
  • [9] Unpaired Image-to-Image Translation via Latent Energy Transport
    Zhao, Yang
    Chen, Changyou
    [J]. 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 16413 - 16422
  • [10] FACE AGING AS IMAGE-TO-IMAGE TRANSLATION USING SHARED-LATENT SPACE GENERATIVE ADVERSARIAL NETWORKS
    Pantraki, Evangelia
    Kotropoulos, Constantine
    [J]. 2018 IEEE GLOBAL CONFERENCE ON SIGNAL AND INFORMATION PROCESSING (GLOBALSIP 2018), 2018, : 306 - 310