Robust quadtree-based disparity estimation for the reconstruction of intermediate stereoscopic images

In stereoscopic/multiview video, the reconstruction of intermediate images is needed to assure continuous motion-parallax and/or comfortable 3-D perception. In this context, we propose a block-based disparity estimation followed by disparity-compensated linear interpolation. We progressively deal with deficiencies of the traditional block matching algorithms. First,we employ a spatial smoothness constraint for disparity to overcome inherent matching ambiguity in low-texture areas. Secondly, as a measure of matching error we use a robust function instead of the quadratic that is sensitive to outliers. We also extend the formulation to include color. Finally, we relax the rigidity of the block support for disparities by employing a quadtree block structure (blocks are allowed to split). The proposed algorithm is implemented in a hierarchical coarse-to-fine fashion with a Gaussian pyramid to reduce the computational burden. To correct luminance and color mismatches between images, a 3-component balancing similar to that proposed by MPEG-2's "Multiview Profile Ad Hoc Group" is used. We tested the proposed algorithm on stereoscopic video sequences acquired in natural surroundings by almost parallel cameras. In informal viewing, every feature of the algorithm listed above resulted in clear improvements of the reconstruction quality. Overall the reconstructed image quality was very good to excellent, depending on the image used.