Multi-spectral stereo image matching using mutual information

Mutual information (MI) has shown promise as an effective stereo matching measure for images affected by radiometric distortion. This is due to the robustness of MI against changes in illumination. However MI-based approaches are particularly prone to the generation of false matches due to the small statistical power of the matching windows. Consequently, most previous MI approaches utilise large matching windows which smooth the estimated disparity field. This paper proposes extensions to MI-based stereo matching in order to increase the robustness of the algorithm. Firstly, prior probabilities are incorporated into the MI measure in order to considerably increase the statistical power of the matching windows. These prior probabilities, which are calculated from the global joint histogram between the stereo pair are tuned to a two level hierarchical approach. A 2D match surface, in which the match score is computed for every possible combination of template and matching window, is also utilised. This enforces left-right consistency and uniqueness constraints. These additions to MI-based stereo matching significantly enhance the algorithm's ability to detect correct matches while decreasing computation time and improving the accuracy. Results show that the MI measure does not perform quite as well for standard stereo pairs when compared to traditional area based metrics. However the MI approach is far superior when matching across multi-spectra stereo pairs.