A Real-time Speech Driven Talking Avatar based on Deep Neural Network

被引：0

作者：

Zhao, Kai ^{[1
]}

Wu, Zhiyong ^{[1
]}

Cai, Lianhong ^{[1
]}

机构：

[1] Tsinghua Univ, Grad Sch Shenzhen, Shenzhen Key Lab Informat Sci & Technol, Tsinghua CUHK Joint Res Ctr Media Sci Technol & S, Shenzhen 518057, Peoples R China

来源：

2013 ASIA-PACIFIC SIGNAL AND INFORMATION PROCESSING ASSOCIATION ANNUAL SUMMIT AND CONFERENCE (APSIPA) | 2013年

关键词：

D O I：

暂无

中图分类号：

TM [电工技术]; TN [电子技术、通信技术];

学科分类号：

0808 ; 0809 ;

摘要：

This paper describes our initial work in developing a real-time speech driven talking avatar system with deep neural network. The input of the system is the acoustic speech and the output is the articulatory movements (that are synchronized with the input speech) on a 3-dimentional avatar. The mapping from the input acoustic features to the output articulatory features is achieved by virtue of deep neural network (DNN). Experiments on the well known acoustic-articulatory English speech corpus MNGU0 demonstrate that the proposed audio-visual mapping method based on DNN can achieve good performance.

引用

页数：4

共 50 条

[1] Real-time speech-driven animation of expressive talking faces
Liu, Jia
You, Mingyu
Chen, Chun
Song, Mingli
INTERNATIONAL JOURNAL OF GENERAL SYSTEMS, 2011, 40 (04) : 439 - 455
[2] Real-Time Speech Enhancement Based on Convolutional Recurrent Neural Network
Girirajan, S.
Pandian, A.
INTELLIGENT AUTOMATION AND SOFT COMPUTING, 2023, 35 (02): : 1987 - 2001
[3] Deep Neural Network Based Real-Time Intrusion Detection System
Sharuka Promodya Thirimanne
Lasitha Jayawardana
Lasith Yasakethu
Pushpika Liyanaarachchi
Chaminda Hewage
SN Computer Science, 2022, 3 (2)
[4] Real-time intraoperative diagnosis by deep neural network driven multiphoton virtual histology
Sixian You
Yi Sun
Lin Yang
Jaena Park
Haohua Tu
Marina Marjanovic
Saurabh Sinha
Stephen A. Boppart
npj Precision Oncology, 3
[5] Real-time intraoperative diagnosis by deep neural network driven multiphoton virtual histology
You, Sixian
Sun, Yi
Yang, Lin
Park, Jaena
Tu, Haohua
Marjanovic, Marina
Sinha, Saurabh
Boppart, Stephen A.
NPJ PRECISION ONCOLOGY, 2019, 3 (1)
[6] Real-time single-channel deep neural network-based speech enhancement on edge devices
Shankar, Nikhil
Bhat, Gautam Shreedhar
Panahi, Issa M. S.
INTERSPEECH 2020, 2020, : 3281 - 3285
[7] WEIGHTED SPEECH DISTORTION LOSSES FOR NEURAL-NETWORK-BASED REAL-TIME SPEECH ENHANCEMENT
Xia, Yangyang
Braun, Sebastian
Reddy, Chandan K. A.
Dubey, Harishchandra
Cutler, Ross
Tashev, Ivan
2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2020, : 871 - 875
[8] Deep Voice: Real-time Neural Text-to-Speech
Arik, Sercan O.
Chrzanowski, Mike
Coates, Adam
Diamos, Gregory
Gibiansky, Andrew
Kang, Yongguo
Li, Xian
Miller, John
Ng, Andrew
Raiman, Jonathan
Sengupta, Shubho
Shoeybi, Mohammad
INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 70, 2017, 70
[9] Real-Time Talking Avatar on the Internet Using Kinect and Voice Conversion
Nose, Takashi
Igarashi, Yuki
INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2015, 6 (12) : 301 - 307
[10] A Convolutional Recurrent Neural Network for Real-Time Speech Enhancement
Tan, Ke
Wang, DeLiang
19TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2018), VOLS 1-6: SPEECH RESEARCH FOR EMERGING MARKETS IN MULTILINGUAL SOCIETIES, 2018, : 3229 - 3233

← 1 2 3 4 5 →