Auditory Event-Related Potentials (ERPs) in Audiovisual Speech Perception

Purpose: It has recently been reported (e.g., V. van Wassenhove, K. W. Grant, & D. Poeppel, 2005) that audiovisual (AV) presented speech is associated with an N1/P2 auditory event-related potential (ERP) response that is lower in peak amplitude compared with the responses associated with auditory only (AO) speech. This effect was replicated. Further comparisons were made between ERP responses to AV speech in which the visual and auditory components were in or out of synchrony, to test whether the effect is associated with the operation of integration mechanisms, as has been claimed, or occurs because of other factors such as attention. Method: ERPs were recorded from participants presented with recordings of unimodal or AV speech syllables in a detection task. Results: Comparisons were made between AO and AV speech and between synchronous and asynchronous AV speech. Synchronous AV speech produced an N1/P2 with lower peak amplitudes than with AO speech, unaccounted for by linear superposition of visually evoked responses onto auditory-evoked responses. Asynchronous AV speech produced no amplitude reduction. Conclusion: The dependence of N1/P2 amplitude reduction on AV synchrony validates it as an electrophysiological marker of AV integration.