On multi-view video segmentation for object-based coding

A novel scheme for multi-view segmentation and tracking is proposed aiming to acquire perceptually consistent results for object-based coding. Firstly, a classic image segmentation technique is employed to perform initial segmentation to divide the whole image into spatially homogeneous regions. Secondly, the motion information is extracted based on frame differences and the disparity information is derived by employing a classic disparity estimation technique. Thirdly, a novel scheme is proposed to perform merging of the initial segmentation results based on both motion and disparity information to remove over-segmented regions and extract perceptually consistent semantic objects. Finally, a contour-based tracking algorithm is proposed to implement accurate and robust object tracking along both temporal and view directions. Experiments are conducted and the results demonstrate that the proposed scheme is effective and, compared with the existing technique, it can acquire more perceptually consistent results. (C) 2012 Elsevier Inc. All rights reserved.