GOYA: Leveraging Generative Art for Content-Style Disentanglement

被引:0
|
作者
Wu, Yankun [1 ]
Nakashima, Yuta [1 ]
Garcia, Noa [1 ]
机构
[1] Osaka Univ, Intelligence & Sensing Lab, Suita, Osaka 5650871, Japan
关键词
art analysis; representation disentanglement; text-to-image generation; CLASSIFICATION;
D O I
10.3390/jimaging10070156
中图分类号
TB8 [摄影技术];
学科分类号
0804 ;
摘要
The content-style duality is a fundamental element in art. These two dimensions can be easily differentiated by humans: content refers to the objects and concepts in an artwork, and style to the way it looks. Yet, we have not found a way to fully capture this duality with visual representations. While style transfer captures the visual appearance of a single artwork, it fails to generalize to larger sets. Similarly, supervised classification-based methods are impractical since the perception of style lies on a spectrum and not on categorical labels. We thus present GOYA, which captures the artistic knowledge of a cutting-edge generative model for disentangling content and style in art. Experiments show that GOYA explicitly learns to represent the two artistic dimensions (content and style) of the original artistic image, paving the way for leveraging generative models in art analysis.
引用
收藏
页数:21
相关论文
共 37 条
  • [1] Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis
    Wu, Yankun
    Nakashima, Yuta
    Garcia, Noa
    [J]. PROCEEDINGS OF THE 2023 ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, ICMR 2023, 2023, : 199 - 208
  • [2] Reproducibility Companion Paper: Stable Diffusion for Content-Style Disentanglement in Art Analysis
    Wu, Yankun
    Nakashima, Yuta
    Garcia, Noa
    Li, Sheng
    Zeng, Zhaoyang
    [J]. PROCEEDINGS OF THE 4TH ANNUAL ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, ICMR 2024, 2024, : 1228 - 1231
  • [3] Leveraging Content-Style Item Representation for Visual Recommendation
    Deldjoo, Yashar
    Di Noia, Tommaso
    Malitesta, Daniele
    Merra, Felice Antonio
    [J]. ADVANCES IN INFORMATION RETRIEVAL, PT II, 2022, 13186 : 84 - 92
  • [4] Unsupervised underwater image enhancement via content-style representation disentanglement
    Zhu, Pengli
    Liu, Yancheng
    Wen, Yuanquan
    Xu, Minyi
    Fu, Xianping
    Liu, Siyuan
    [J]. ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2023, 126
  • [5] Style and Content Disentanglement in Generative Adversarial Networks
    Kazemi, Hadi
    Iranmanesh, Seyed Mehdi
    Nasrabadi, Nasser M.
    [J]. 2019 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2019, : 848 - 856
  • [6] Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translation
    Kwon, Gihyun
    Ye, Jong Chul
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 13960 - 13969
  • [7] Density-Aware Haze Image Synthesis by Self-Supervised Content-Style Disentanglement
    Zhang, Chi
    Lin, Zihang
    Xu, Liheng
    Li, Zongliang
    Tang, Wei
    Liu, Yuehu
    Meng, Gaofeng
    Wang, Le
    Li, Li
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2022, 32 (07) : 4552 - 4572
  • [8] Content and Style Disentanglement for Artistic Style Transfer
    Kotovenko, Dmytro
    Sanakoyeu, Artsiom
    Lang, Sabine
    Ommer, Bjorn
    [J]. 2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 4421 - 4430
  • [9] Fine-grained Style Modeling, Transfer and Prediction in Text-to-Speech Synthesis via Phone-Level Content-Style Disentanglement
    Tan, Daxin
    Lee, Tan
    [J]. INTERSPEECH 2021, 2021, : 4683 - 4687
  • [10] CoSMo: Content-Style Modulation for Image Retrieval with Text Feedback
    Lee, Seungmin
    Kim, Dongwan
    Han, Bohyung
    [J]. 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 802 - 812