Correction Compensation and Adaptive Cost Aggregation for Deep Laparoscopic Stereo Matching

Featured Application This work explores the computation of disparity in stereoscopic laparoscopic images through stereo matching algorithms. By integrating the focal length and baseline of the laparoscopic vision system, we can transform the disparity into depth measurements. This digitized depth information facilitates the three-dimensional reconstruction of surgical scenes, and the real-time three-dimensional reconstructed images have the potential to provide supplementary guidance information to surgeons during procedures, thereby reducing surgical risks. Additionally, by leveraging this known digitized depth information, surgical robots can synchronize their movements with beating organs, thus reducing the complexity of such surgeries.Abstract Perception of digitized depth is a prerequisite for enabling the intelligence of three-dimensional (3D) laparoscopic systems. In this context, stereo matching of laparoscopic stereoscopic images presents a promising solution. However, the current research in this field still faces challenges. First, the acquisition of accurate depth labels in a laparoscopic environment proves to be a difficult task. Second, errors in the correction of laparoscopic images are prevalent. Finally, laparoscopic image registration suffers from ill-posed regions such as specular highlights and textureless areas. In this paper, we make significant contributions by developing (1) a correction compensation module to overcome correction errors; (2) an adaptive cost aggregation module to improve prediction performance in ill-posed regions; (3) a novel self-supervised stereo matching framework based on these two modules. Specifically, our framework rectifies features and images based on learned pixel offsets, and performs differentiated aggregation on cost volumes based on their value. The experimental results demonstrate the effectiveness of the proposed modules. On the SCARED dataset, our model reduces the mean depth error by 12.6% compared to the baseline model and outperforms the state-of-the-art unsupervised methods and well-generalized models.