FIGI: floating interface for gesture-based interaction

Mixed reality represents a promising technology for a wide range of applicative fields, including computer based training, systems maintenance and medical imaging, just to name a few. The floating interface for gesture-based interaction architecture presented in this study, puts together a context adaptive head-up interface, which is projected in the central region of the user’s visual field, with gesture-based interaction, to enable easy, robust and powerful manipulation of the virtual contents which are visualized after being mapped onto the real environment surrounding the user. The interaction paradigm combines one-hand, two-hands and time-based gestures to select tools/functions among those available as well as to operate them. Even conventional keyboard-based functions like typing, can be performed without a physical interface by means of a floating keyboard layout. The paper describes the overall system architecture and its application to the interactive visualization of tri-dimensional models of human anatomy, for either training or educational purposes. We also report the results of an evaluation study to assess usability, effectiveness and eventual limitations of the proposed approach.