Virtual Reality-Based Time-Delayed Haptic Teleoperation Using Point Cloud Data

This paper presents a methodology for bilateral teleoperation of robots based on the use of point cloud data obtained with an RGB-D camera for providing haptic feedback, as well as a virtual robot workspace that allows the user to interact with the teleoperated robot and the environment, regardless of the time-delay between the master device and the slave robot. The robot movements are executed first in the Virtual Workspace and then in the real one. The Virtual Workspace is created using point cloud data, and the haptic feedback is calculated directly using this data. The proposed methodology was validated in 2D and 3D trajectory-following experiments with 20 volunteers each teleoperating a KUKA industrial robot. In addition, we studied two inverse kinematic teleoperation modes: “Proportional Workspace” and “Delta end-effector movements”, with the objective of determining which one is more suitable for the haptic teleoperation methodology proposed. The results obtained confirm that the proposed haptic teleoperation methodology enhances the precision of the operator by reducing the average error between the robot end-effector trajectory and the goal trajectory. Furthermore, the likelihood that the end-effector will collide with objects in the environment is reduced. The methodology also diminishes the time that it takes to complete the trajectory-following task in an unknown workspace. The inverse kinematic teleoperation mode with the better results is the “Proportional Workspace”, which is more intuitive for operators (users).