Multimodal human-computer interaction system for speech rehabilitation

被引：0
|
作者：
面向言语康复的多模态人机交互系统
机构：
[1] [1,Zhao, Jian
[2] 2,Wang, Liu
[3] 2,Shi, Li-Juan
[4] 1,Kuang, Zhe-Jun
[5] Wang, Si-Bo
[6] 1,Sun, Yuan-Yuan
来源：
Shi, Li-Juan (shilj@ccu.edu.cn) | 1600年 / Editorial Board of Jilin University卷 / 50期
关键词：
Visual communication - Human computer interaction - Patient rehabilitation;
D O I：
10.13229/j.cnki.jdxbgxb20190941
中图分类号：
学科分类号：
摘要：
In order to solve the problem that the vibration of vocal cords can not be objectively reflected by touch and subjective evaluation in the process of speech rehabilitation training, a multi-modal human-computer interaction system based on video-audio feedback and vibration feedback is proposed in this paper. The system uses three-dimensional virtual head image to realize audio-video feedback and vibration motor to realize vocal cords vibration. Multimodal human-computer interaction training with dynamic feedback can assist hearing-impaired children to complete pronunciation rehabilitation training. The experimental results show that the system and device are effective in improving the effect of speech rehabilitation training for hearing-impaired children. © 2020, Jilin University Press. All right reserved.
引用
收藏
页码：1478 / 1486
相关论文
共 50 条
  • [31] Design and implementation of human-computer interaction intelligent system based on speech control
    Liu, Jichao
    Chang, Wenhui
    Li, Jing
    Wang, Ju
    [J]. Computer-Aided Design and Applications, 2020, 17 (Special Issue 2) : 22 - 34
  • [32] An experimental study of input modes for multimodal human-computer interaction
    Ren, XH
    Zhang, G
    Dai, GH
    [J]. ADVANCES IN MULTIMODAL INTERFACES - ICMI 2000, PROCEEDINGS, 2000, 1948 : 49 - 56
  • [33] USER REPRESENTATIONS OF COMPUTER-SYSTEMS IN HUMAN-COMPUTER SPEECH INTERACTION
    AMALBERTI, R
    CARBONELL, N
    FALZON, P
    [J]. INTERNATIONAL JOURNAL OF MAN-MACHINE STUDIES, 1993, 38 (04): : 547 - 566
  • [34] Towards a Universal Human-Computer Interaction Model for Multimodal Interactions
    Faltaous, Sarah
    Gruenefeld, Uwe
    Schneegass, Stefan
    [J]. MENSCH AND COMPUTER 2021 (MUC 21), 2021, : 59 - 63
  • [35] Multimodal human-computer interfaces
    Dutoit, Thierry
    Nigay, Laurence
    Schnaider, Michael
    [J]. SIGNAL PROCESSING, 2006, 86 (12) : 3515 - 3517
  • [36] Modeling multimodal human-computer interaction: Semiotics, proxemics and kinesics
    Marshall, R
    [J]. DESIGN OF COMPUTING SYSTEMS: SOCIAL AND ERGONOMIC CONSIDERATIONS, 1997, 21 : 671 - 674
  • [37] A Multimodal Human-Computer Interaction for Smart Learning System (May 2023, 10.1080/10447318.2023.2206758 )
    Alzubi, T. M.
    Alzubi, J. A.
    Singh, A.
    Alzubi, O. A.
    Subramanian, M.
    [J]. INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION, 2023,
  • [38] Toward an affect-sensitive multimodal human-computer interaction
    Pantic, M
    Rothkrantz, LJM
    [J]. PROCEEDINGS OF THE IEEE, 2003, 91 (09) : 1370 - 1390
  • [39] InfoPlant: Multimodal augmentation of plants for enhanced human-computer interaction
    Hammerschmidt, Jan
    Hermann, Thomas
    Walender, Alex
    Kroemker, Niels
    [J]. 2015 6TH IEEE INTERNATIONAL CONFERENCE ON COGNITIVE INFOCOMMUNICATIONS (COGINFOCOM), 2015, : 511 - 516
  • [40] Exploiting Unconscious User Signals in Multimodal Human-Computer Interaction
    Andre, Elisabeth
    [J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2013, 9 (01)
12345