Reaching Motion Planning with Vision-Based Deep Neural Networks for Dual Arm Robots

Dual arm robots have been attracting attention from the view point of factory automation. These robots are basically required to reach their hands toward the respective target objects, simultaneously. Therefore, we focus on motion planning with vision-based deep neural networks. Given an RGB-D camera mounted on a robot, object images are fed as the inputs to the reaching motion planner based on convolutional neural network, CNN. For multiple objects, the depth of each object in the image is useful information to determine a reaching target. If the objects are close to each other, however, the depth becomes similar. For this challenge, we propose to generate the target object image through instance segmentation and order classifier. In the experiment with multiple objects, we show that the robot is able to reach both the hands toward the target objects by using the target object images.