Semi-automatic Synchronization of Sensory Effects in Mulsemedia Authoring Tools

Synchronization of sensory effects with multimedia content is a non-trivial and error-prone task that can discourage authoring of mulsemedia applications. Although there are authoring tools that assist in the specification of sensory effect metadata in an automated way, the forms of analysis used by them are not general enough to identify complex components that may be related to sensory effects. In this work, we present an intelligent component, which allows the semi-automatic definition of sensory effects. This component uses a neural network to extract information from video scenes. This information is used to set sensory effects synchronously to related videos. The proposed component was implemented in STEVE 2.0 authoring tool, helping the authoring of sensory effects in a graphical interface.