Robot End-Effector Mounted Camera Pose Optimization in Object Detection-Based Tasks

Robots equipped with the vision systems at the end-effector provide a powerful combination in industrial contexts, allowing to execute a wide range of manufacturing tasks, such as inspection applications. While many works are dedicated to machine vision algorithms, the optimization of the vision system pose is not properly addressed. Optimizing the sensor pose, in fact, can increase the object detection performance, avoiding occlusions and collisions in the real working scene. Therefore, the development of an approach capable of optimizing the pose of a vision system is the main objective of this paper. A complete pipeline for such optimization is proposed, composed of the following main components: working scene reconstruction, robot-environment collisions modeling, object detection, sensor pose optimization (exploiting Bayesian Optimization, a state of the art methodology), and collision-free robot motion planning. To validate the proposed approach, experimental tests have been executed considering two object detection-based tasks. A Franka EMIKA Panda robot equipped with an Intel(C) RealSense D400 at its end-effector has been employed as a robotic platform. Achieved results show the high-fidelity reconstruction of the real working environment for an offline optimization (i.e., performed simulations), as well as the capabilities of the employed Bayesian Optimization-based approach to define the sensor pose. The proposed optimization methodology has been compared with the grid point approach, showing an improved performance for camera pose optimization purposes. An additional experiment has been performed in order to show the possibility to exploit a digital twin (if available) of the working scene instead of the environment reconstruction (to reduce the computational resources and to avoid measurements noise in the 3D reconstruction). Obtained results show the feasibility of the proposed pipeline employing such a digital twin.