Exploiting Multimodal Interfaces in e-Learning Systems

Nowadays, eLearning becomes an important way to transfer knowledge to learners. Thus, the number of eLearning content (eLearning websites, digital books, tutorials) has increased significantly during the last decade. However, the majority of the available eLearning solutions interact with the user through the traditional Human Computer Interface, using mainly, texts and graphics to communicate the information which overload the visual channel of the user. Therefore, the communicated information may be missed, as well as the focus and interest of the user can decrease in time. Multimodal interactions turn the communication between the user and the machine in a more natural way using different channels that humans usually use to communicate between them. Integrating a Multimodal Interface to any eLearning module will make the delivery of the information easier and more efficient. It will also ensure the delivery of the communicated information and improves the users' focus and interests during the sessions. This paper investigates the use and effects of a multimodal interface in an eLearning module, part of a complex system, that helps users to optimize the benefits of the system and assists them in case of any problem. The investigation involves two different types of interfaces. The first type is a textual interface based on text and graphics, with the traditional input modalities. The second type is based on multimodal metaphors and integrates three input modalities: speech, gesture and touch, beside the traditional ones; and two output modalities: visual and phonetic. We propose a testing platform for comparing the two types of interfaces, that will track different aspects in the user-machine interactions: the testing platform will track the number of the interactions in each session, the emotional status of the user, the time needed by the user to accomplish a task, the task difficulty and the obtained results of each session... All these results will be enhanced with information collected through a questionnaire that will be filled by the user after each session. In order to ensure a thorough understanding in the eLearning module, all the collected data will be used to create a user profile in order to customize the interface using user's preferences and needs, which will optimize the experience of the user and increase the quality of life.