Multi-camera Coordinate Calibration and Accuracy Evaluation for Robot Control

In this study, our goal is to create a surgical navigation system that takes stable measurements of a surgical area using RGB/depth cameras without obstructions, such as the surgeon's head or hands. We mounted three cutting-edge D435 Intel RealSense Depth Cameras onto a ring and photographed the surgical area from three directions. We also installed a robotic mechanical system that can move the camera ring up and down so that the surgery can proceed smoothly. First, we calibrated the coordinate systems so that the coordinate systems of the three cameras (three-dimensional XYZ coordinate system) align and their Y-axis (vertical axis) aligns with the moving axis of the robot slider. Next, we captured an ArUco marker with each camera and visualized its position within the camera coordinate system. After verifying that the initial positions of the ArUco marker captured by the three cameras match, we moved the robot slider up by 50 mm thrice and investigated the degree of change in the measured position of the ArUco marker measured by the three cameras. The results show that as the ArUco marker moves farther away, the extent of error in the measured position of the ArUco marker increases. Additionally, the measured position of all the ArUco markers varied owing to the digital pixel error in the two-dimensional images (the pixel in which the ArUco marker is visible moves between neighboring pixels). Future work includes checking that the calibration pattern and ArUco marker are parallel to the camera ring and perpendicular to the robot slider using a level instrument; packing the calibration pattern and ArUco marker horizontally with vinyl; and implementing a moving average in the program to reduce pixel digital error in the calculations to sub-pixel level.