Multimodal human-computer interaction system for speech rehabilitation

被引：0
|
作者：
Zhao, Jian [1 ,2 ]
Wang, Liu [2 ,3 ]
Shi, Li-Juan [2 ,3 ]
Kuang, Zhe-Jun [1 ,2 ]
Wang, Si-Bo [3 ]
Sun, Yuan-Yuan [1 ,3 ]
机构：
[1] College of Computer Science and Technology, Changchun University, Changchun,130022, China
[2] Jilin Provincial Key Laboratory of Human Health Status Identification and Function Enhancement, Changchun University, Changchun,130022, China
[3] College of Electronic Information Engineering, Changchun University, Changchun,130022, China
来源：
Jilin Daxue Xuebao (Gongxueban)/Journal of Jilin University (Engineering and Technology Edition) | 2020年 / 50卷 / 04期
关键词：
Audition;
D O I：
10.13229/j.cnki.jdxbgxb20190941
中图分类号：
学科分类号：
摘要：
In order to solve the problem that the vibration of vocal cords can not be objectively reflected by touch and subjective evaluation in the process of speech rehabilitation training, a multi-modal human-computer interaction system based on video-audio feedback and vibration feedback is proposed in this paper. The system uses three-dimensional virtual head image to realize audio-video feedback and vibration motor to realize vocal cords vibration. Multimodal human-computer interaction training with dynamic feedback can assist hearing-impaired children to complete pronunciation rehabilitation training. The experimental results show that the system and device are effective in improving the effect of speech rehabilitation training for hearing-impaired children. © 2020, Jilin University Press. All right reserved.
引用
收藏
页码：1478 / 1486
相关论文
共 50 条
  • [1] Speech and language processing for multimodal human-computer interaction
    Deng, L
    Wang, Y
    Wang, K
    Acero, A
    Hon, H
    Droppo, J
    Boulis, C
    Mahajan, M
    Huang, XD
    [J]. JOURNAL OF VLSI SIGNAL PROCESSING SYSTEMS FOR SIGNAL IMAGE AND VIDEO TECHNOLOGY, 2004, 36 (2-3): : 161 - 187
  • [2] Speech and Language Processing for Multimodal Human-Computer Interaction
    L. Deng
    Y. Wang
    K. Wang
    A. Acero
    H. Hon
    J. Droppo
    C. Boulis
    M. Mahajan
    X.D. Huang
    [J]. Journal of VLSI signal processing systems for signal, image and video technology, 2004, 36 : 161 - 187
  • [3] Speech timing prediction in multimodal human-computer interaction
    Bourguet, ML
    Ando, A
    [J]. HUMAN-COMPUTER INTERACTION - INTERACT '97, 1997, : 453 - 460
  • [4] Speech recognition and intelligent translation under multimodal human-computer interaction system
    Huang, Danhua
    Xiang, Shuaiqiu
    [J]. JOURNAL OF INTELLIGENT SYSTEMS, 2024, 33 (01)
  • [5] Multimodal human-computer interaction
    Turk, M
    [J]. REAL-TIME VISION FOR HUMAN-COMPUTER INTERACTION, 2005, : 269 - 283
  • [6] A Multimodal Human-Computer Interaction for Smart Learning System
    Alzubi, Tareq Mahmod
    Alzubi, Jafar A.
    Singh, Ashish
    Alzubi, Omar A.
    Subramanian, Murali
    [J]. INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION, 2023,
  • [7] Multimodal Interfaces of Human-Computer Interaction
    Karpov, A. A.
    Yusupov, R. M.
    [J]. HERALD OF THE RUSSIAN ACADEMY OF SCIENCES, 2018, 88 (01) : 67 - 74
  • [8] Considerations on Multimodal Human-Computer Interaction
    Liu, Chang
    Xie, Wenjun
    Zhang, Peng
    Zhan, Jia
    Xiao, Zonghao
    [J]. PROCEEDINGS OF 2018 5TH IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS (CCIS), 2018, : 331 - 335
  • [9] MULTIMODAL COLLABORATION AND HUMAN-COMPUTER INTERACTION
    Zhang, Zhengyou
    [J]. ICME: 2009 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOLS 1-3, 2009, : 1596 - 1599
  • [10] Multimodal human-computer interaction: A survey
    Jaimes, Alejandro
    Sebe, Nicu
    [J]. COMPUTER VISION AND IMAGE UNDERSTANDING, 2007, 108 (1-2) : 116 - 134
12345