Concept and Architecture for Programming Industrial Robots using Augmented Reality with Mobile Devices like Microsoft HoloLens

This paper proposes a concept for human-robot interaction using techniques of virtual and augmented reality on mobile devices like cell phones and tablets or mixed reality devices like the HoloLens. By combining data received from real robots together with the perception and abilities of a human operator innovative applications are imaginable. Visualizing not only the current robot state but also the robots environment captured with different sensors and processed with both machine and human vision can lead to a rising percentage of robot assisted workplaces or robot installations. Since the visualization of and the interaction with the robotic application is not locally restricted new or improved use cases like remote maintenance or faster startup of industrial robots can be realized. Therefore, an architecture split into three functional units and an overview of our implementation is presented.