Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech

被引：9

作者：

Kucherenko, Taras ^{[1
]}

Nagy, Rajmund ^{[2
]}

Jonell, Patrik ^{[2
]}

Neff, Michael ^{[3
]}

Kjellstrom, Hedvig ^{[1
]}

Henter, Gustav Eje ^{[2
]}

机构：

[1] KTH Royal Inst Technol, Robot Percept & Learning, Stockholm, Sweden

[2] KTH Royal Inst Technol, Speech Mus & Hearing, Stockholm, Sweden

[3] Univ Calif Davis, Davis, CA 95616 USA

来源：

PROCEEDINGS OF THE 21ST ACM INTERNATIONAL CONFERENCE ON INTELLIGENT VIRTUAL AGENTS (IVA) | 2021年

关键词：

gesture generation; virtual agents; representational gestures;

D O I：

10.1145/3472306.3478333

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures. Our approach first predicts whether to gesture, followed by a prediction of the gesture properties. Those properties are then used as conditioning for a modern probabilistic gesture-generation model capable of high-quality output. This empowers the approach to generate gestures that are both diverse and representational. Follow-ups and more information can be found on the project page: https://svito-zar.github.io/speech2properties2gestures/

引用

页码：145 / 147

页数：3

共 21 条

[1] Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders
Li, Jing
Kang, Di
Pei, Wenjie
Zhe, Xuefei
Zhang, Ying
He, Zhenyu
Bao, Linchao
[J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 11273 - 11282
[2] Representational disfluency in algebra: Evidence from student gestures and speech
Bieda K.N.
Nathan M.J.
[J]. ZDM, 2009, 41 (5): : 637 - 650
[3] Representational gestures correlated with meaning-associated aspects of L2 speech performance
Ma, Sai
Jin, Guangsa
Barlow, Michael
[J]. GESTURE, 2021, 20 (03) : 376 - 416
[4] Audio2Gestures: Generating Diverse Gestures From Audio
Li, Jing
Kang, Di
Pei, Wenjie
Zhe, Xuefei
Zhang, Ying
Bao, Linchao
He, Zhenyu
[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2024, 30 (08) : 4752 - 4766
[5] Gesture2Vec: Clustering Gestures using Representation Learning Methods for Co-speech Gesture Generation
Yazdian, Payam Jome
Chen, Mo
Lim, Angelica
[J]. 2022 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2022, : 3100 - 3107
[6] The Role of Representational Gestures and Speech Synchronicity in Auditory Input by L2 and L1 Speakers
Federica Cavicchio
Maria Grazia Busà
[J]. Journal of Psycholinguistic Research, 2023, 52 : 1721 - 1735
[7] The Role of Representational Gestures and Speech Synchronicity in Auditory Input by L2 and L1 Speakers
Cavicchio, Federica
Busa, Maria Grazia
[J]. JOURNAL OF PSYCHOLINGUISTIC RESEARCH, 2023, 52 (05) : 1721 - 1735
[8] Generating Co -Speech Gestures for Virtual Agents from Multimodal Information Based on Transformer
Yu, Yue
Shi, Jiande
[J]. 2023 IEEE CONFERENCE ON VIRTUAL REALITY AND 3D USER INTERFACES ABSTRACTS AND WORKSHOPS, VRW, 2023, : 887 - 888
[9] Gesture style can affect the integration of gestures and speech: the evidence from Chinese ERP research
Sun, Fang
Xiang, Huiwen
Hu, Xinzhuo
Li, Yutong
Sui, Xue
[J]. NEUROREPORT, 2020, 31 (12) : 885 - 890
[10] The relationship between different types of co-speech gestures and L2 speech performance
Ma, Sai
Jin, Guangsa
[J]. FRONTIERS IN PSYCHOLOGY, 2022, 13

← 1 2 3 →