Walking-in-place for omnidirectional VR locomotion using a single RGB camera

Locomotion is a fundamental interaction element allowing navigation inside the virtual environment, and the walking-in-place (WIP) techniques have been actively developed as a balanced compromise between naturalness and efficiency. One popular method to implement the WIP technique was to use a low-cost, easy to set up, and markerless Kinect, but required integration of multiple sensors or covered limited directions due to the poor tracking capability when facing non-frontal sides of the user. This study aimed to propose a WIP technique for omnidirectional VR locomotion based on a single RGB camera, utilizing an open-source 2D human pose estimation system called OpenPose. Three WIP techniques (existing Kinect-based technique, proposed Kinect-based technique, and proposed OpenPose-based technique) were compared in terms of variation of virtual walking speed and subjective evaluation through a user study with walking tasks in different directions. Experimental results showed that the proposed OpenPose-based technique performed comparably when the user faced the front of the camera, but it induced lower variation of virtual walking speed and higher subjective evaluation ratings at non-forward directions compared to other techniques. The proposed OpenPose-based WIP technique can be used in VR applications to provide a fully unobstructed VR locomotion experience. It can achieve stable WIP-based omnidirectional VR locomotion through a single low-cost easily accessible RGB camera, without the need for additional sensors, and at the same time, both hands are free for other interactions.