Multimodal human-computer interaction: A survey

被引：438

作者：

Jaimes, Alejandro ^{[1
]}

Sebe, Nicu ^{[1
]}

机构：

[1] Univ Amsterdam, NL-1012 WX Amsterdam, Netherlands

来源：

COMPUTER VISION AND IMAGE UNDERSTANDING | 2007年 / 108卷 / 1-2期

关键词：

multimodal human-computer interaction; gesture recognition; affective computing; human-centered computing;

D O I：

10.1016/j.cviu.2006.10.019

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

In this paper, we review the major approaches to multimodal human-computer interaction, giving an overview of the field from a computer vision perspective. In particular, we focus on body, gesture, gaze, and affective interaction (facial expression recognition and emotion in audio). We discuss user and task modeling, and multimodal fusion, highlighting challenges, open issues, and emerging applications for multimodal human-computer interaction (MMHCI) research. (C) 2007 Elsevier Inc. All rights reserved.

引用

页码：116 / 134

页数：19

共 50 条

[1] Multimodal human-computer interaction
Turk, M
[J]. REAL-TIME VISION FOR HUMAN-COMPUTER INTERACTION, 2005, : 269 - 283
[2] Multimodal Interfaces of Human-Computer Interaction
Karpov, A. A.
Yusupov, R. M.
[J]. HERALD OF THE RUSSIAN ACADEMY OF SCIENCES, 2018, 88 (01) : 67 - 74
[3] Considerations on Multimodal Human-Computer Interaction
Liu, Chang
Xie, Wenjun
Zhang, Peng
Zhan, Jia
Xiao, Zonghao
[J]. PROCEEDINGS OF 2018 5TH IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS (CCIS), 2018, : 331 - 335
[4] MULTIMODAL COLLABORATION AND HUMAN-COMPUTER INTERACTION
Zhang, Zhengyou
[J]. ICME: 2009 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOLS 1-3, 2009, : 1596 - 1599
[5] Modeling multimodal human-computer interaction
Obrenovic, Z
Starcevic, D
[J]. COMPUTER, 2004, 37 (09) : 65 - +
[6] Systematic Review of Multimodal Human-Computer Interaction
Azofeifa, Jose Daniel
Noguez, Julieta
Ruiz, Sergio
Molina-Espinosa, Jose Martin
Magana, Alejandra J.
Benes, Bedrich
[J]. INFORMATICS-BASEL, 2022, 9 (01):
[7] Measuring Multimodal Synchrony for Human-Computer Interaction
Reidsma, Dennis
Nijholt, Anton
Tschacher, Wolfgang
Ramseyer, Fabian
[J]. 2010 INTERNATIONAL CONFERENCE ON CYBERWORLDS (CW 2010), 2010, : 67 - 71
[8] Gaze tracking for multimodal human-computer interaction
Stiefelhagen, R
Yang, J
[J]. 1997 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOLS I - V: VOL I: PLENARY, EXPERT SUMMARIES, SPECIAL, AUDIO, UNDERWATER ACOUSTICS, VLSI; VOL II: SPEECH PROCESSING; VOL III: SPEECH PROCESSING, DIGITAL SIGNAL PROCESSING; VOL IV: MULTIDIMENSIONAL SIGNAL PROCESSING, NEURAL NETWORKS - VOL V: STATISTICAL SIGNAL AND ARRAY PROCESSING, APPLICATIONS, 1997, : 2617 - 2620
[9] Human-computer intelligent interaction: A survey
Lew, Michael
Bakker, Erwin M.
Sebe, Nicu
Huang, Thomas S.
[J]. HUMAN-COMPUTER INTERACTION, PROCEEDINGS, 2007, 4796 : 1 - +
[10] Multimodal sentence similarity in human-computer interaction systems
Ferri, Fernando
Grifoni, Patrizia
Paolozzi, Stefano
[J]. KNOWLEDGE-BASED INTELLIGENT INFORMATION AND ENGINEERING SYSTEMS: KES 2007 - WIRN 2007, PT II, PROCEEDINGS, 2007, 4693 : 403 - 410

← 1 2 3 4 5 →