Multimodal Affective Pedagogical Agents for Different Types of Learners

The paper reports progress on an NSF-funded project whose goal is to research and develop multimodal affective animated pedagogical agents (APA) for different types of learners. Although the preponderance of research on APA tends to focus on the cognitive aspects of online learning, this project explores the less-studied role of affective features. More specifically, the objectives of the work are to: (1) research and develop novel algorithms for emotion recognition and for life-like emotion representation in embodied agents, which will be integrated in a new system for creating APA to be embedded in digital lessons; and (2) develop an empirically grounded research base that will guide the design of affective APA that are effective for different types of learners. This involves conducting a series of experiments to determine the effects of the agent's emotional style and emotional intelligence on a diverse population of students. The paper outlines the work conducted so far, e.g., development of a new system (and underlying algorithms) for producing affective APA. It also reports the findings from two preliminary studies.