The contribution of the visual modality to vowel perception in native and non-native speakers

This study examines how visual cues impact the intelligibility of foreign-accented speech for native listeners. English CVC words with vowels involving salient (e.g., lip-spreading for /i:/) and non-salient visual cues (neutral lips for /I/) produced by two French speakers and a native English control, were presented to native English listeners who identified the word heard. Tokens were presented in both auditory-only and audiovisual (AV) mode in cafeteria noise at 15 dB SNR. The visual cues analysed were lip spreading, lip rounding, jaw opening and tongue frontness in vowels, as well as lip-rounding in schwa. Visually salient cues improved vowel intelligibility, compared to non-visual cues, but the audiovisual benefit varied across vowel features and speaker groups. The presence of lip-spreading for /i:/ (vs /I/) and jaw-opening for /a=/ (vs /I/) enhanced intelligibility (i.e., larger AV benefit) for both speaker groups. However, compared to the English speaker, lip-rounding in /(sic):/ and /(sic)/ produced by French speakers (likely accompanied by lip protrusion) had a smaller, or negative AV benefit. These results suggest that the influence of L1 gestures on L2 production may reduce or negatively affect intelligibility. Furthermore, French productions of /alpha:/ exhibited unusually high AV benefits, suggesting an extreme jaw-opening for this vowel in an attempt to distinguish between L2 contrasts (/ae boolean AND alpha:/) not present in the L1. (c) 2024 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.