Exploiting Semantic and Boundary Information for Stereo Matching

Stereo matching aims to estimating disparity by finding the correspondence of each pixel between two images which is crucial to 3D scene reconstruction. Nowadays 3D convolution neural networks achieve impressive performances on stereo matching. However, it is memory consuming and computation complex. And it is challenging to finding the corresponding pixels in textureless and near boundary regions. Therefore, a stereo matching neural network is proposed which use semantic segmentation and boundary detection task to improve the accuracy of stereo matching near boundary and textureless regions. And a hybrid cost volume which reflects the similarity between left and right feature map, is designed to contains semantic cost volume and boundary cost volume with attention mechanism. The stereo matching neural network is designed to rely on coarse-to-fine strategy which predict a complete disparity map at the highest resolution and refine disparity at the lower resolution. We conduct comprehensive experiments on KITTI 2015 datasets, and compare with some recent stereo matching neural networks, the D1-all (3-pixel error) is 2.8% and run time is 0.044s which shows that embedding semantic and boundary information can improve the accuracy of stereo matching.