Disentangling latent space better for few-shot image-to-image translation

被引:2
|
作者
Liu, Peng [1 ]
Wang, Yueyue [1 ]
Du, Angang [2 ]
Zhang, Liqiang [2 ]
Wei, Bin [4 ,5 ]
Gu, Zhaorui [2 ]
Wang, Xiaodong [3 ]
Zheng, Haiyong [2 ]
Li, Juan [6 ]
机构
[1] Ocean Univ China, Comp Ctr, Qingdao 266100, Peoples R China
[2] Ocean Univ China, Dept Elect Engn, Qingdao 266100, Peoples R China
[3] Ocean Univ China, Dept Comp Sci & Technol, Qingdao 266100, Peoples R China
[4] Qingdao Univ, Affiliated Hosp, Qingdao 266000, Peoples R China
[5] Shandong Key Lab Digital Med & Comp Assisted Surg, Qingdao 266000, Peoples R China
[6] Qingdao Agr Univ, Coll Mech & Elect Engn, Qingdao 266000, Peoples R China
基金
中国国家自然科学基金;
关键词
Image-to-image translation; Generative adversarial network; Latent space; Few-shot learning; Disentanglement;
D O I
10.1007/s13042-022-01552-4
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In an unpaired image-to-image translation, the main concept is to learn an underlying mapping between the source and target domains. Previous approaches required large numbers of data from both domains to learn this mapping. However, under a few-shot condition, that is, few-shot image-to-image translation, only one domain can meet the required number of data , and thus, the underlying mapping becomes ill-conditioned owing to the limited data as well as the imbalanced distribution of the two domains. We argue that a powerful model with a better disentangled representation of the latent space can better tackle the more challenging few-shot image-to-image translation . Motivated by this, under a partially-shared assumption, we propose a better disentanglement of the content and style latent space using a domain-specific style latent classifier and a domain-shared cross-content latent discriminator. Moreover, we design asymmetric weak/strong domain discriminators to achieve a better translation performance with limited data within the few-shot domain. Furthermore, our method can be easily embedded into any latent space disentangled model of an image-to-image translation for a few-shot setting. Subjective evaluation and objective evaluation results both show that compared with other state-of-the-art methods, the images synthesized by our method have higher fidelity while maintaining certain diversity.
引用
收藏
页码:419 / 427
页数:9
相关论文
共 50 条
  • [1] Disentangling latent space better for few-shot image-to-image translation
    Peng Liu
    Yueyue Wang
    Angang Du
    Liqiang Zhang
    Bin Wei
    Zhaorui Gu
    Xiaodong Wang
    Haiyong Zheng
    Juan Li
    International Journal of Machine Learning and Cybernetics, 2023, 14 : 419 - 427
  • [2] Few-Shot Unsupervised Image-to-Image Translation
    Liu, Ming-Yu
    Huang, Xun
    Mallya, Arun
    Karras, Tero
    Aila, Timo
    Lehtinen, Jaakko
    Kautz, Jan
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 10550 - 10559
  • [3] Semi-supervised Learning for Few-shot Image-to-Image Translation
    Wang, Yaxing
    Khan, Salman
    Gonzalez-Garcia, Abel
    van de Weijer, Joost
    Khan, Fahad Shahbaz
    2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, : 4452 - 4461
  • [4] Learning a Latent Space with Triplet Network for Few-Shot Image Classification
    Wu, Jiaying
    Hu, Jinglu
    2022 26TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2022, : 5038 - 5044
  • [5] WeditGAN: Few-Shot Image Generation via Latent Space Relocation
    Duan, Yuxuan
    Niu, Li
    Hong, Yan
    Zhang, Liqing
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 2, 2024, : 1653 - 1661
  • [6] Homomorphic Latent Space Interpolation for Unpaired Image-To-Image Translation
    Chen, Ying-Cong
    Xu, Xiaogang
    Tian, Zhuotao
    Jia, Jiaya
    2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 2403 - 2411
  • [7] OA-FSUI2IT: A Novel Few-Shot Cross Domain Object Detection Framework with Object-Aware Few-Shot Unsupervised Image-to-Image Translation
    Zhao, Lifan
    Meng, Yunlong
    Xu, Lin
    THIRTY-SIXTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FOURTH CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE / THE TWELVETH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2022, : 3426 - 3435
  • [8] Few-Shot Face Sketch-to-Photo Synthesis via Global-Local Asymmetric Image-to-Image Translation
    Li, Yongkang
    Liang, Qifan
    Han, Zhen
    Mai, Wenjun
    Wang, Zhongyuan
    ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2024, 20 (10)
  • [9] Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation
    Liu, Yahui
    Sangineto, Enver
    Chen, Yajing
    Bao, Linchao
    Zhang, Haoxian
    Sebe, Nicu
    Lepri, Bruno
    Wang, Wei
    De Nadai, Marco
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 10780 - 10789
  • [10] ManiFest: Manifold Deformation for Few-Shot Image Translation
    Pizzati, Fabio
    Lalonde, Jean-Francois
    de Charette, Raoul
    COMPUTER VISION - ECCV 2022, PT XVII, 2022, 13677 : 440 - 456