Region-based motion estimation for content-based video coding and indexing

For the past decade, the region-based approach, that combines object segmentation and optical flow estimation, has emerged as the only one likely to provide automatically, at a reasonable computational cost, higher-quality descriptions of 2D apparent motion in video sequences, as compared to conventional pixel-based motion estimation. Within this framework, a hybrid algorithm, embedding classical dense motion field estimation and color-based spatial segmentation, is presented. Per each, arbitrarily shaped, color-homogeneous region, a polynomial motion-parameter set is robustly estimated from pixel displacement vectors. Following a graph-based approach and starting from the initial color partition, neighboring regions are iteratively merged according to their mutual motion similarity. The obtained motion-homogeneous regions are eventually temporally tracked along the sequence. The region-based motion estimation algorithm is described in details and its computational complexity is loosely evaluated through processing time statistics on a workstation. The partition maps and modeled motion fields obtained on three well-known test sequences - "Table Tennis", "Mobile and calendar" and "Flower Garden" - are displayed. Alternative approaches in the literature are then assessed, their results being compared with the above ones. Application of such an automatic "mid-level" image analysis tool to object-based representation, manipulation and coding as well as indexing of video is outlined at last.