Visual-Haptic Colocation in Robotic Rehabilitation Exercises Using a 2D Augmented-Reality Display

Haptics-based Virtual Reality (VR) games have been found to be effective in rehabilitation from disability. Augmented Reality (AR) has gained traction in recent years in various domains including gaming, entertainment, and education. In this paper, we integrate spatial AR into robotic rehabilitation to provide colocation between visual and haptic feedback as a human user participates in a rehabilitative game. A comparison between the effectiveness of VR vs AR (i.e., non-colocation vs colocation of vision) is done. Spatial AR is the colocation of vision through the use of projection. Visual-Haptic colocation is the combination of spatial AR and haptic interaction. We also compare each visualization technique in the absence and presence of haptic feedback and cognitive loading (CL) for the human user. The system was evaluated by having 10 able-bodied participants do all 8 different conditions lasting approximately 3 minutes per condition. The results show that spatial AR (corresponding to colocation of visual frame and hand frame) leads to the best user performance when doing the task regardless of the presence or the absence of haptics. It is also observed that for users undergoing cognitive loading, the combination of spatial AR and haptics produces the best result in terms of task completion time.