Enabling Embodied Conversational Agents to Respond to Nonverbal Behavior of the Communication Partner

Humans communicate on three levels: words, paralanguage, and nonverbal. While conversational agents focus mainly on the interpretation of words that are being spoken, recently the focus has also shifted to how we say those words with our tone, pace, and intonation. Nonverbal communication, including facial expression, eye contact, posture, and proximity, has been largely ignored in human-agent interactions. In this work, we propose to incorporate nonverbal behavior into conversations between humans and agents by displaying a human-like embodied agent on a large screen and by responding appropriately to nonverbal cues from the interlocutors. In a user study with 19 volunteers, we investigated the influence on the participants for different behaviors (mimicry, positively biased mimicry, negatively biased mimicry, random) of the embodied conversation agents. The results indicate that goaldirected behavior is perceived significantly better concerning likability, social competence, attitude, and responsiveness in comparison to random behavior. This indicates that already simple nonverbal methods of building rapport can be used to improve the perceived conversational quality with an embodied conversational agent.