Construction Robot Operation System with Object's Hardness Recognition Using Force Feedback and Virtual Reality

We developed a master-slave operation system for a teleoperation construction robot that recognizes the hardness of a grasped object. To manipulate an object, the system uses an excavator with four degrees of freedom as a slave and two joysticks with force feedback equipment as a master. Based on creating a friendly user interface, the operation system uses multimodel sensory force and visual feedback to successfully discriminate among soft object types during operation. The construction robot measures the hardness of an object using the pressure of a piston obtained by pressure sensors on the cylinder and the closed or open state of a fork glove in the process of grasping an object. By incorporating an object-hardness calculation method with master-slave control of the system, an operator then can feel the sense of reaction force to joysticks and distinguish the hardness of an object while controlling the construction robot. In addition, parameters on object-hardness calculation are presented to the operator to improve the system's controllability. Color prompting is provided in virtual space to enable the operator to identify the hardness of an object. To evaluate the system, object-type recognition tests were conducted, including the grasping and conveying of blocks of concrete, tires, urethane foam and sponge foam. According to statistical analysis of experimental results, we confirmed that the operation system contributes to achieving the successful discrimination of object hardness during teleoperation work.