An augmented reality-based training system with a natural user interface for manual milling operations

This study developed an augmented reality (AR)-based training system for conventional manual milling operations. An Intel RealSense R200 depth camera and a Leap Motion controller were mounted on an HTC Vive head-mounted display to allow users freely walk around in a room-size AR environment to operate a full-size virtual milling machine with their barehands, using their natural operation behaviors, as if they were operating a real milling machine in the real world, without additional worn or handheld devices. GPU parallel computing was used to handle dynamic occlusions and accelerate the machining simulation to achieve a real-time simulation. Using the developed AR-based training system, novices can receive a hands-on training in a safe environment, without any injury or damage. User test results showed that using the developed AR-based training resulted in lower failure rates and inquiry times than using video training. Users also commented that the AR-based training was interesting and helpful for novices to learn the basic manual milling operation techniques.