Review of Multimodal Interaction in Optical See-Through Augmented Reality

The development of AR OST-HMDs has expanded interaction possibilities beyond traditional HCI, fostering innovation in MMI. However, the lack of research specifically investigating the combined effects of different modalities, encompassing both input and output in AR OST-HMD, impedes a comprehensive understanding of this research intersection. This study bridges this gap by reviewing empirical studies on MMI in AR OST-HMDs conducted from 2012 to 2022, identifying the current state of research, and synthesizing knowledge on the implementation and evaluation of MMI in this context. The analysis of 30 empirical research yielded insights into the current usage and combinations of input and output modalities in AR OST-HMDs, its known effects, and the approaches used to study and evaluate them. We have identified that input and output are mostly examined as independent components, overlooking the potential effects of input-output combinations. We present opportunities and recommendations for future research in MMI in AR OST-HMD.