Speech driven facial animation

被引：0
|
作者：
Yang, TJ [1 ]
Lin, IC [1 ]
Hung, CS [1 ]
Huang, CF [1 ]
Ming, OY [1 ]
机构：
[1] Natl Taiwan Univ, Dept Comp Sci & Informat Engn, Communicat & Multimedia Lab, Taipei 106, Taiwan
来源：
COMPUTER ANIMATION AND SIMULATION'99 | 1999年
关键词：
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
In this paper, we present an approach that animates facial expressions through speech analysis. An individualized 3D head model is first generated by modifying a generic head model, where a set of MPEG-4 Facial Definition Parameters (FDPs) has been pre-defined. To animate facial expressions of the 3D head model, a real-time speech analysis module is employed to obtain mouth shapes that are converted to MPEG-4 Facial Animation Parameters (FAPs) to drive the 3D head model with corresponding facial expressions. The approach has been implemented as a real-time speech-driven facial animation system. On a PC with a single Pentinum-III 500MHz CPU, the system performance is around 15-24 frames/sec with image size 120x150. The input is live audio, and initial delay is within 4 seconds. An ongoing model-based visual communication system that integrates a 3D head motion estimation technique with this system is also described.
引用
收藏
页码：99 / 108
页数：10
相关论文
共 50 条
  • [31] Speech-Driven 3D Facial Animation with Mesh Convolution
    Ji, Xuejie
    Su, Zewei
    Dong, Lanfang
    Li, Guoming
    [J]. 2022 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, COMPUTER VISION AND MACHINE LEARNING (ICICML), 2022, : 14 - 18
  • [32] Speech Driven Tongue Animation
    Medina, Salvador
    Tome, Denis
    Stoll, Carsten
    Tiede, Mark
    Munhall, Kevin
    Hauptmann, Alex
    Matthews, Iain
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 20374 - 20384
  • [33] An approach to speech driven animation
    Sun, Ningping
    Suigetsu, Kaori
    Ayabe, Toru
    [J]. 2008 FOURTH INTERNATIONAL CONFERENCE ON INTELLIGENT INFORMATION HIDING AND MULTIMEDIA SIGNAL PROCESSING, PROCEEDINGS, 2008, : 113 - 116
  • [34] CLTalk: Speech-Driven 3D Facial Animation with Contrastive Learning
    Zhang, Xitie
    Wu, Suping
    [J]. PROCEEDINGS OF THE 4TH ANNUAL ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, ICMR 2024, 2024, : 1175 - 1179
  • [35] Pose-Aware Speech Driven Facial Landmark Animation Pipeline for Automated Dubbing
    Bigioi, Dan
    Jordan, Hugh
    Jain, Rishabh
    McDonnell, Rachel
    Corcoran, Peter
    [J]. IEEE ACCESS, 2022, 10 (133357-133369) : 133357 - 133369
  • [36] Geometry-Guided Dense Perspective Network for Speech-Driven Facial Animation
    Liu, Jingying
    Hui, Binyuan
    Li, Kun
    Liu, Yunke
    Lai, Yu-Kun
    Zhang, Yuxiang
    Liu, Yebin
    Yang, Jingyu
    [J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2022, 28 (12) : 4873 - 4886
  • [37] Audio-to-Visual Conversion Via HMM Inversion for Speech-Driven Facial Animation
    Terissi, Lucas D.
    Gomez, Juan Carlos
    [J]. ADVANCES IN ARTIFICIAL INTELLIGENCE - SBIA 2008, PROCEEDINGS, 2008, 5249 : 33 - 42
  • [38] Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animation
    Fu, Hui
    Wang, Zeqing
    Gong, Ke
    Wang, Keze
    Chen, Tianshui
    Li, Haojie
    Zeng, Haifeng
    Kang, Wenxiong
    [J]. THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 2, 2024, : 1770 - 1777
  • [39] Speech driven photo realistic facial animation based on an articulatory DBN model and AAM features
    Dongmei Jiang
    Yong Zhao
    Hichem Sahli
    Yanning Zhang
    [J]. Multimedia Tools and Applications, 2014, 73 : 397 - 415
  • [40] Speech-Driven Facial Animation Using a Shared Gaussian Process Latent Variable Model
    Deena, Salil
    Galata, Aphrodite
    [J]. ADVANCES IN VISUAL COMPUTING, PT 1, PROCEEDINGS, 2009, 5875 : 89 - 100
12345