Dual Contrastive Learning for Unsupervised Image-to-Image Translation

被引:108
|
作者
Han, Junlin [1 ,2 ]
Shoeiby, Mehrdad [1 ]
Petersson, Lars [1 ]
Armin, Mohammad Ali [1 ]
机构
[1] DATA61 CSIRO, Canberra, ACT, Australia
[2] Australian Natl Univ, Canberra, ACT, Australia
关键词
D O I
10.1109/CVPRW53098.2021.00084
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Unsupervised image-to-image translation tasks aim to find a mapping between a source domain X and a target domain Y from unpaired training data. Contrastive learning for Unpaired image-to-image Translation (CUT) yields state-of-the-art results in modeling unsupervised image-to-image translation by maximizing mutual information between input and output patches using only one encoder for both domains. In this paper, we propose a novel method based on contrastive learning and a dual learning setting (exploiting two encoders) to infer an efficient mapping between unpaired data. Additionally, while CUT suffers from mode collapse, a variant of our method efficiently addresses this issue. We further demonstrate the advantage of our approach through extensive ablation studies demonstrating superior performance comparing to recent approaches in multiple challenging image translation tasks. Lastly, we demonstrate that the gap between unsupervised methods and supervised methods can be efficiently closed.
引用
收藏
页码:746 / 755
页数:10
相关论文
共 50 条
  • [41] Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Tasks
    Jung, Chanyong
    Kwon, Gihyun
    Ye, Jong Chul
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 18239 - 18248
  • [42] Dual contrastive learning based image-to-image translation of unstained skin tissue into virtually stained H&E images
    Asaf, Muhammad Zeeshan
    Rao, Babar
    Akram, Muhammad Usman
    Khawaja, Sajid Gul
    Khan, Samavia
    Truong, Thu Minh
    Sekhon, Palveen
    Khan, Irfan J.
    Abbasi, Muhammad Shahmir
    [J]. SCIENTIFIC REPORTS, 2024, 14 (01)
  • [43] Unaligned Image-to-Image Translation by Learning to Reweight
    Xie, Shaoan
    Gong, Mingming
    Xu, Yanwu
    Zhang, Kun
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 14154 - 14164
  • [44] Heterogeneous Image Change Detection Based on Dual Image Translation and Dual Contrastive Learning
    Ma, Zongfang
    Wang, Ruiqi
    Hao, Fan
    Song, Lin
    [J]. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024, 62 : 1 - 14
  • [45] Underwater dam crack image generation based on unsupervised image-to-image translation
    Huang, Ben
    Kang, Fei
    Li, Xinyu
    Zhu, Sisi
    [J]. AUTOMATION IN CONSTRUCTION, 2024, 163
  • [46] Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation
    Liu, Yahui
    Sangineto, Enver
    Chen, Yajing
    Bao, Linchao
    Zhang, Haoxian
    Sebe, Nicu
    Lepri, Bruno
    Wang, Wei
    De Nadai, Marco
    [J]. 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 10780 - 10789
  • [47] UNSUPERVISED IMAGE-TO-IMAGE TRANSLATION VIA FAIR REPRESENTATI ON OF GENDER BIAS
    Hwang, Sunhee
    Byun, Hyeran
    [J]. 2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2020, : 1953 - 1957
  • [48] Instance-wise Hard Negative Example Generation for Contrastive Learning in Unpaired Image-to-Image Translation
    Wang, Weilun
    Zhou, Wengang
    Bao, Jianmin
    Chen, Dong
    Li, Houqiang
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 14000 - 14009
  • [49] Retrieval Guided Unsupervised Multi-domain Image-to-Image Translation
    Gomez, Raul
    Liu, Yahui
    De Nadai, Marco
    Karatzas, Dimosthenis
    Lepri, Bruno
    Sebe, Nicu
    [J]. MM '20: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, 2020, : 3164 - 3172
  • [50] SPatchGAN: A Statistical Feature Based Discriminator for Unsupervised Image-to-Image Translation
    Shao, Xuning
    Zhang, Weidong
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 6526 - 6535