Bi-Window Based Stereo Matching Using Combined Siamese Convolutional Neural Network

Estimating a scene's depth from image pairs is a problem for many computer vision applications such as autonomous vehicles and 3D object reconstruction. However, traditional methods propose dissimilarity functions that do not solve ambiguity problems. We present a combined siamese convolutional neural network (CSCNN) approach to calculate the costs of disparities in stereo images using patches with different sizes. This approach can learn the patches context, improving the accuracy of the estimated costs. We apply the semi-global matching method and the median filter to increase further the robustness of matching costs. We trained and evaluated our approach using the Middlebury database, and, through the bad pixel evaluation, we demonstrate that our approach achieved an accuracy improvement of approximately 22%, compared to the results obtained by single-window CNN-based approaches.