Language-driven nonverbal communication in a bilingual conversational agent

This paper describes an animated conversational agent called Karel which integrates a talking head interface with a linguistically motivated human-machine dialogue system. The agent has a range of nonverbal behaviors, which involve a mixture of machine vision, computer animation and natural language processing techniques. The system's architecture couples the agent's nonverbal communicative processes very tightly to its model of verbal interaction. We discuss several consequences of this architecture, in particular the ability to use different non-verbal dialogue management signals when speaking different languages.