Multimodal user interfaces in ubiquitous sensorised environments

This paper aims to show where ubiquitous sensor environments in combination with multimodal user interfaces can be used to further enhance the user (and usage) experience. The challenge in multimodal user interface design is to combine the available user input modalities with the available sensor data in a meaningful way. A conceptual framework for sensorised modality input composition and semantically controlled output decomposition in Ubiquitous Sensorised Environments is introduced and its individual components, the Ubiquitous Sensorised Environment, the input processing component, the situation aware application component and the output preparation component are described. The paper also illustrates the functionality by means of an p example, namely user mood awareness and application. Finally considerable focus is on the sensors nodes, classed in to independent, parasitic and viral sensors and some examples of what sensor payloads need to be employed to facilitate mood sensing are given.