3-D structure assisted reference view generation for H.264 based multi-view video coding

A 3D geometry-based multi-view video coding (MVC) method is proposed. In order to utilize the spatial redundancies between multiple views, the scene geometry is estimated as dense depth maps. The dense depth estimation problem is modeled by using a Markov random field (MRF) and solved via the belief propagation algorithm. Relying on these depth maps of the scene, novel view estimates of the intermediate views of the multi-view set is obtained with a 3D warping algorithm, which also performs hole-filling in the occlusion regions. The proposed MVC method, based on H.264 standard, encodes a number of reference views in a standard manner, whereas the residuals of the novel view predictions are encoded separately. The proposed MVC method is compared against the well-known JMVM compression algorithm, yielding competitive performances, while additionally providing 3D structure information of the observed scene.