The importance of gaze and gesture in interactive multimodal explanation

被引：10
|
作者：
Lund, Kristine [1 ]
机构：
[1] Univ Lyon, CNRS, ICAR, Ecole Normale Super Lettres & Sci Humaines, F-69342 Lyon, France
来源：
LANGUAGE RESOURCES AND EVALUATION | 2007年 / 41卷 / 3-4期
关键词：
collaboration; computer-mediated human interaction; interactive explanation; gaze; gesture; multimodal communication;
D O I：
10.1007/s10579-007-9058-0
中图分类号：
TP39 [计算机的应用];
学科分类号：
081203 ; 0835 ;
摘要：
The objective of this research is twofold. Firstly, we argue that gaze and gesture play an essential part in interactive explanation and that it is thus a multimodal phenomenon. Two corpora are analyzed: (1) a group of teacher novices and experts and (2) a student teacher dyad, both of whom construct explanations of students' reasoning after viewing videos of student dyads who are solving physics problems. We illustrate roles of gaze in explanations constructed within a group and roles of gesture in explanation constructed within a dyad. Secondly, we show how the analysis of such knowledge-rich empirical data pinpoints particular difficulties in designing human-computer interfaces that can support explanation between humans, or a fortiori, that can support explanation between a human and a computer.
引用
收藏
页码：289 / 303
页数：15
相关论文
共 50 条
  • [1] The importance of gaze and gesture in interactive multimodal explanation
    Kristine Lund
    [J]. Language Resources and Evaluation, 2007, 41 : 289 - 303
  • [2] Multimodal Interaction with Gaze and Controller Gesture
    Chia, Wen Han
    Cai, Yiyu
    Ho, Andrew
    [J]. 2022 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY ADJUNCT (ISMAR-ADJUNCT 2022), 2022, : 518 - 523
  • [3] Multimodal Driver Interaction with Gesture, Gaze and Speech
    Aftab, Abdul Rafey
    [J]. ICMI'19: PROCEEDINGS OF THE 2019 INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION, 2019, : 487 - 492
  • [4] A Multimodal System for Gesture Recognition in Interactive Music Performance
    Overholt, Dan
    Thompson, John
    Putnam, Lance
    Bell, Bo
    Kleban, Jim
    Sturm, Bob
    Kuchera-Morin, Joann
    [J]. COMPUTER MUSIC JOURNAL, 2009, 33 (04) : 69 - 82
  • [5] A gesture-driven multimodal interactive dance system
    Qian, G
    Guo, F
    Ingalls, T
    Olson, L
    James, J
    Rikakis, T
    [J]. 2004 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXP (ICME), VOLS 1-3, 2004, : 1579 - 1582
  • [6] Role of the interactive gesture and gaze in regulating responsive by maritally satisfied and unsatisfied couples
    Bélanger, I
    Simard, MC
    Lavoie, S
    Chartrand, É
    [J]. CANADIAN JOURNAL OF BEHAVIOURAL SCIENCE-REVUE CANADIENNE DES SCIENCES DU COMPORTEMENT, 2002, 34 (04): : 250 - 260
  • [7] Gesture, gaze, and ground
    McNeill, D
    [J]. MACHINE LEARNING FOR MULTIMODAL INTERACTION, 2005, 3869 : 1 - 14
  • [8] Multimodal Segmentation on a Large Interactive Tabletop: Extending Interaction on Horizontal Surfaces with Gaze
    Newn, Joshua
    Velloso, Eduardo
    Carter, Marcus
    Vetere, Frank
    [J]. PROCEEDINGS OF THE 2016 ACM INTERNATIONAL CONFERENCE ON INTERACTIVE SURFACES AND SPACES, (ISS 2016), 2016, : 251 - 260
  • [9] What you see is what you do: on the relationship between gaze and gesture in multimodal alignment
    Oben, Bert
    Brone, Geert
    [J]. LANGUAGE AND COGNITION, 2015, 7 (04) : 546 - 562
  • [10] Gaze and Gesture Activity in Communication
    Jokinen, Kristiina
    [J]. UNIVERSAL ACCESS IN HUMAN-COMPUTER INTERACTION, PT II, PROCEEDINGS: INTELLIGENT AND UBIQUITOUS INTERACTION ENVIRONMENTS, 2009, 5615 : 537 - 546
12345