Dual Contrastive Learning for Unsupervised Image-to-Image Translation

被引:108
|
作者
Han, Junlin [1 ,2 ]
Shoeiby, Mehrdad [1 ]
Petersson, Lars [1 ]
Armin, Mohammad Ali [1 ]
机构
[1] DATA61 CSIRO, Canberra, ACT, Australia
[2] Australian Natl Univ, Canberra, ACT, Australia
关键词
D O I
10.1109/CVPRW53098.2021.00084
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Unsupervised image-to-image translation tasks aim to find a mapping between a source domain X and a target domain Y from unpaired training data. Contrastive learning for Unpaired image-to-image Translation (CUT) yields state-of-the-art results in modeling unsupervised image-to-image translation by maximizing mutual information between input and output patches using only one encoder for both domains. In this paper, we propose a novel method based on contrastive learning and a dual learning setting (exploiting two encoders) to infer an efficient mapping between unpaired data. Additionally, while CUT suffers from mode collapse, a variant of our method efficiently addresses this issue. We further demonstrate the advantage of our approach through extensive ablation studies demonstrating superior performance comparing to recent approaches in multiple challenging image translation tasks. Lastly, we demonstrate that the gap between unsupervised methods and supervised methods can be efficiently closed.
引用
收藏
页码:746 / 755
页数:10
相关论文
共 50 条
  • [1] Contrastive learning for unsupervised image-to-image translation
    Lee, Hanbit
    Seol, Jinseok
    Lee, Sang-goo
    Park, Jaehui
    Shim, Junho
    [J]. APPLIED SOFT COMPUTING, 2024, 151
  • [2] Truly Unsupervised Image-to-Image Translation with Contrastive Representation Learning
    Hong, Zhiwei
    Feng, Jianxing
    Jiang, Tao
    [J]. COMPUTER VISION - ACCV 2022, PT III, 2023, 13843 : 239 - 255
  • [3] DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
    Yi, Zili
    Zhang, Hao
    Tan, Ping
    Gong, Minglun
    [J]. 2017 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2017, : 2868 - 2876
  • [4] Spectral normalization and dual contrastive regularization for image-to-image translation
    Zhao, Chen
    Cai, Wei-Ling
    Yuan, Zheng
    [J]. VISUAL COMPUTER, 2024,
  • [5] Improving Learning time in Unsupervised Image-to-Image Translation
    Min, Tae-Hong
    Kim, Do-Yun
    Choi, Young-June
    [J]. 2019 1ST INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE IN INFORMATION AND COMMUNICATION (ICAIIC 2019), 2019, : 455 - 458
  • [6] Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation
    Lin, Yupei
    Zhang, Sen
    Chen, Tianshui
    Lu, Yongyi
    Li, Guangping
    Shi, Yukai
    [J]. PROCEEDINGS OF THE 30TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2022, 2022, : 1186 - 1194
  • [7] Multimodal Unsupervised Image-to-Image Translation
    Huang, Xun
    Liu, Ming-Yu
    Belongie, Serge
    Kautz, Jan
    [J]. COMPUTER VISION - ECCV 2018, PT III, 2018, 11207 : 179 - 196
  • [8] Unsupervised Image-to-Image Translation Networks
    Liu, Ming-Yu
    Breuel, Thomas
    Kautz, Jan
    [J]. ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 30 (NIPS 2017), 2017, 30
  • [9] Unsupervised Image-to-Image Translation: A Review
    Hoyez, Henri
    Schockaert, Cedric
    Rambach, Jason
    Mirbach, Bruno
    Stricker, Didier
    [J]. SENSORS, 2022, 22 (21)
  • [10] Perceptual Contrastive Generative Adversarial Network based on image warping for unsupervised image-to-image translation
    Huang, Lin-Chieh
    Tsai, Hung-Hsu
    [J]. NEURAL NETWORKS, 2023, 166 : 313 - 325