The role of sensory-motor modality compatibility in language processing

Language processing requires the combination of compatible (auditory-vocal and visual-manual) or incompatible (auditory-manual and visual-vocal) sensory-motor modalities, and switching between these sensory-motor modality combinations is very common in every-day life. Sensory-motor modality compatibility is defined as the similarity of stimulus modality and the modality of response-related sensory consequences. We investigated the influence of sensory-motor modality compatibility during performing language-related cognitive operations on different linguistic levels. More specifically, we used a variant of the task-switching paradigm, in which participants had to switch between compatible or between incompatible sensory-motor modality combinations during a verbal semantic categorization (Experiment 1) or during a word-form decision (Experiment 2). The data show higher switch costs (i.e., higher reaction times and error rates in switch trials compared to repetition trials) in incompatible sensory-motor modality combinations than in compatible sensory-motor modality combinations. This was true for every language-related cognitive operation, regardless of the individual linguistic level. Taken together, the present study demonstrates that sensory-motor modality compatibility plays an important role in modality switching during language processing.