SMooDi: Stylized Motion Diffusion Model

被引:0
|
作者
Zhong, Lei [1 ]
Xie, Yiming [1 ]
Jampani, Varun [2 ]
Sun, Deqing [3 ]
Jiang, Huaizu [1 ]
机构
[1] Northeastern Univ, Boston, MA 02115 USA
[2] Stabil AI, London, England
[3] Google Res, Mountain View, CA USA
来源
关键词
Motion synthesis; Diffusion model; Stylized motion;
D O I
10.1007/978-3-031-73232-4_23
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We introduce a novel Stylized Motion Diffusion model, dubbed SMooDi, to generate stylized motion driven by content texts and style motion sequences. Unlike existing methods that either generate motion of various content or transfer style from one sequence to another, SMooDi can rapidly generate motion across a broad range of content and diverse styles. To this end, we tailor a pre-trained text-to-motion model for stylization. Specifically, we propose style guidance to ensure that the generated motion closely matches the reference style, alongside a lightweight style adaptor that directs the motion towards the desired style while ensuring realism. Experiments across various applications demonstrate that our proposed framework outperforms existing methods in stylized motion generation. Project Page: https://neu-vi.github.io/SMooDi/
引用
收藏
页码:405 / 421
页数:17
相关论文
共 50 条
  • [1] Stylized Liquid Motion Effects on Illustrations
    Fujita, Kyosuke
    Morimoto, Yuki
    Journal of the Institute of Image Electronics Engineers of Japan, 2022, 51 (04): : 332 - 337
  • [2] Sign Motion Generation by Motion Diffusion Model
    Hakozaki, Kohei
    Murakami, Tomoya
    Uchida, Tsubasa
    Miyazaki, Taro
    Kaneko, Hiroyuki
    PROCEEDINGS OF THE SIGGRAPH 2024 POSTERS, 2024,
  • [3] Autoregressive Stylized Motion Synthesis with Generative Flow
    Wen, Yu-Hui
    Yang, Zhipeng
    Fu, Hongbo
    Gao, Lin
    Sun, Yanan
    Liu, Yong-Jin
    2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 13607 - 13616
  • [4] WDANet: Exploring Stylized Animation via Diffusion Model for Woodcut-Style Design
    Ou, Yangchunxue
    Xu, Jingjun
    COMPUTER ANIMATION AND VIRTUAL WORLDS, 2025, 36 (01)
  • [5] Diffusion in a model for active Brownian motion
    Condat, CA
    Sibona, GJ
    PHYSICA D-NONLINEAR PHENOMENA, 2002, 168 : 235 - 243
  • [6] SMEAR: Stylized Motion Exaggeration with ARt-direction
    Basset, Jean
    Benard, Pierre
    Barla, Pascal
    PROCEEDINGS OF SIGGRAPH 2024 CONFERENCE PAPERS, 2024,
  • [7] Stylized Motion Generalization Through Adaptation of Velocity Profiles
    Gielniak, Michael J.
    Liu, C. Karen
    Thomaz, Andrea L.
    2010 IEEE RO-MAN, 2010, : 304 - 309
  • [8] Solutions to a model for interface motion by interface diffusion
    Alber, Hans-Dieter
    Zhu, Peicheng
    PROCEEDINGS OF THE ROYAL SOCIETY OF EDINBURGH SECTION A-MATHEMATICS, 2008, 138 : 923 - 955
  • [9] Skew Brownian motion: A model for diffusion with interfaces?
    Cantrell, RS
    Cosner, C
    MATHEMATICAL MODELS IN MEDICAL AND HEALTH SCIENCE, 1998, : 73 - 78
  • [10] Minimal model of financial stylized facts
    Delpini, Danilo
    Bormetti, Giacomo
    PHYSICAL REVIEW E, 2011, 83 (04):