Processing of changes in visual speech in the human auditory cortex

被引：109
|
作者：
Möttönen, R [1 ]
Krause, CM [1 ]
Tiippana, K [1 ]
Sams, M [1 ]
机构：
[1] Helsinki Univ Technol, Lab Computat Engn, FIN-02150 Espoo, Finland
来源：
COGNITIVE BRAIN RESEARCH | 2002年 / 13卷 / 03期
基金：
芬兰科学院;
关键词：
audiovisual; auditory evoked responses; information processing; lipreading; McGurk effect; mismatch field; multisensory;
D O I：
10.1016/S0926-6410(02)00053-8
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Seeing a talker's articulatory gestures may affect the observer's auditory speech percept. Observing congruent articulatory gestures may enhance the recognition of speech sounds [J. Acoust. Soc. Am. 26 (1954) 212], whereas observing incongruent gestures may change the auditory percept phonetically as occurs in the McGurk effect [Nature 264 (1976) 746]. For example, simultaneous acoustic /ba/ and visual /ga/ are usually heard as /da/. We studied cortical processing of occasional changes in audiovisual and visual speech stimuli with magnetoencephalography. In the audiovisual experiment congruent (acoustic /iti/, visual /iti/) and incongruent (acoustic /ipi/, visual /iti/) audiovisual stimuli, which were both perceived as /iti/, were presented among congruent /ipi/ (acoustic /ipi/, visual /ipi/) stimuli. In the visual experiment only the visual components of these stimuli were presented. A visual change both in audiovisual and visual experiments activated supratemporal auditory cortices bilaterally. The auditory cortex activation to a visual change occurred later in the visual than in the audiovisual experiment, suggesting that interaction between modalities accelerates the detection of visual change in speech. (C) 2002 Elsevier Science B.V. All rights reserved.
引用
收藏
页码：417 / 425
页数：9
相关论文
共 50 条
  • [1] Hierarchical processing for speech in human auditory cortex and beyond
    Peelle, Jonathan E.
    Johnsrude, Ingrid S.
    Davis, Matthew H.
    [J]. FRONTIERS IN HUMAN NEUROSCIENCE, 2010, 4
  • [2] Processing of music and speech by the human auditory cortex: Neuroimaging evidence
    Zatorre, RJ
    [J]. BRAIN AND COGNITION, 2004, 54 (02) : 129 - 129
  • [3] Processing of pitch and location in human auditory cortex during visual and auditory tasks
    Hakkinen, Suvi
    Ovaska, Noora
    Rinne, Teemu
    [J]. FRONTIERS IN PSYCHOLOGY, 2015, 6
  • [4] A comparison of visual and auditory motion processing in human cerebral cortex
    Lewis, JW
    Beauchamp, MS
    DeYoe, EA
    [J]. CEREBRAL CORTEX, 2000, 10 (09) : 873 - 888
  • [5] Bimodal speech: early suppressive visual effects in human auditory cortex
    Besle, J
    Fort, A
    Delpuech, C
    Giard, MH
    [J]. EUROPEAN JOURNAL OF NEUROSCIENCE, 2004, 20 (08) : 2225 - 2234
  • [6] Auditory speech processing is affected by visual speech in the periphery
    Kim, Jeesun
    Davis, Chris
    [J]. 12TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION 2011 (INTERSPEECH 2011), VOLS 1-5, 2011, : 2476 - 2479
  • [7] Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing
    Rauschecker, Josef P.
    Scott, Sophie K.
    [J]. NATURE NEUROSCIENCE, 2009, 12 (06) : 718 - 724
  • [8] Context modulates processing of speech sounds in the right auditory cortex of human subjects
    Kujala, A
    Alho, K
    Valle, S
    Sivonen, P
    Ilmoniemi, RJ
    Alku, P
    Näätänen, R
    [J]. NEUROSCIENCE LETTERS, 2002, 331 (02) : 91 - 94
  • [9] Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing
    Josef P Rauschecker
    Sophie K Scott
    [J]. Nature Neuroscience, 2009, 12 : 718 - 724
  • [10] The effect of visual speech on auditory cortical processing
    Davis, C.
    Kislyuh, D.
    Sams, M.
    Kim, J.
    [J]. AUSTRALIAN JOURNAL OF PSYCHOLOGY, 2006, 58 : 71 - 71
12345