DMDIT: Diverse multi-domain image-to-image translation

被引:7
|
作者
Shao, Mingwen [1 ]
Zhang, Youcai [1 ]
Liu, Huan [1 ]
Wang, Chao [1 ]
Li, Le [1 ]
Shao, Xun [2 ]
机构
[1] China Univ Petr, Coll Comp Sci & Technol, Qingdao, Peoples R China
[2] Fujian Normal Univ, Sch Phys & Energy, Fuzhou, Peoples R China
基金
中国国家自然科学基金;
关键词
Multi-domain; Multi-modality; Image translation; GENERATIVE ADVERSARIAL NETWORKS;
D O I
10.1016/j.knosys.2021.107311
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Cross-domain image translation studies have shown brilliant progress in recent years, which intend to learn the mapping between two different domains. A good cross-domain image translation model should meet the following conditions: (1) do not rely on paired dataset, (2) can deal with multiple domains, (3) obtain diverse outputs with the same source image. Most state-of-art studies are devoted to addressing two of them i.e., either (1) and (2), or (1) and (3). In this paper, we construct a unified diverse multi-domain image to image translation framework (DMDIT) which can satisfy the above three requirements simultaneously. Different from traditional approaches, the proposed generator can achieve diverse and multi-label image-to-image translation while retaining the underlying features of the input image. The diverse outputs are obtained through a latent noise sampled from the normal distribution randomly. To further improve the multiplicity of the outputs, we propose a novel style regularization loss to restrain the latent noise. The mode collapse problem usually occurs due to the lack of constraints on the noise, so we embed a noise separation module in the discriminator to avoid this issue. In addition, we apply an attention mechanism to make the model attentively focus on the most attribute-relevant regions, helping to improve the quality of the generated images. Extensive qualitative and quantitative evaluations clearly demonstrate the effectiveness of our approach. (C) 2021 Published by Elsevier B.V.
引用
收藏
页数:9
相关论文
共 50 条
  • [1] Multi-Domain Image-to-Image Translation with Adaptive Inference Graph
    The-Phuc Nguyen
    Lathuiliere, Stephane
    Ricci, Elisa
    [J]. 2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), 2021, : 5368 - 5375
  • [2] Retrieval Guided Unsupervised Multi-domain Image-to-Image Translation
    Gomez, Raul
    Liu, Yahui
    De Nadai, Marco
    Karatzas, Dimosthenis
    Lepri, Bruno
    Sebe, Nicu
    [J]. MM '20: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, 2020, : 3164 - 3172
  • [3] Multi-Domain Image-to-Image Translation via a Unified Circular Framework
    Wang, Yuxi
    Zhang, Zhaoxiang
    Hao, Wangli
    Song, Chunfeng
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 : 670 - 684
  • [4] Cross-Granularity Learning for Multi-Domain Image-to-Image Translation
    Fu, Huiyuan
    Yu, Ting
    Wang, Xin
    Ma, Huadong
    [J]. MM '20: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, 2020, : 3099 - 3107
  • [5] RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributes
    Wu, Po-Wei
    Lin, Yu-Jing
    Chang, Che-Han
    Chang, Edward Y.
    Liao, Shih-Wei
    [J]. 2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 5913 - 5921
  • [6] Self-attention StarGAN for Multi-domain Image-to-Image Translation
    He, Ziliang
    Yang, Zhenguo
    Mao, Xudong
    Lv, Jianming
    Li, Qing
    Liu, Wenyin
    [J]. ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2019: IMAGE PROCESSING, PT III, 2019, 11729 : 537 - 549
  • [7] MULTI-DOMAIN UNSUPERVISED IMAGE-TO-IMAGE TRANSLATION WITH APPEARANCE ADAPTIVE CONVOLUTION
    Jeong, Somi
    Lee, Jiyoung
    Sohn, Kwanghoon
    [J]. 2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2022, : 1750 - 1754
  • [8] Dual Generator Generative Adversarial Networks for Multi-domain Image-to-Image Translation
    Tang, Hao
    Xu, Dan
    Wang, Wei
    Yan, Yan
    Sebe, Nicu
    [J]. COMPUTER VISION - ACCV 2018, PT I, 2019, 11361 : 3 - 21
  • [9] Multi-Domain Image-to-Image Translation with Cross-Granularity Contrastive Learning
    Fu, Huiyuan
    Liu, Jin
    Yu, Ting
    Wang, Xin
    Ma, Huadong
    [J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2024, 20 (07)
  • [10] StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
    Choi, Yunjey
    Choi, Minje
    Kim, Munyoung
    Ha, Jung-Woo
    Kim, Sunghun
    Choo, Jaegul
    [J]. 2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, : 8789 - 8797