Motion estimation, segmentation and separation, using hypercomplex phase correlation, clustering techniques and graph-based optimization

The problem of estimating the motions in image sequences has been extensively studied. However, most of the proposed motion estimation approaches deal only with the luminance component of the images, although the color information is also important. The usual way to apply these techniques in color image sequences is to process each color channel separately. As proposed by Ell, Sangwine and Moxey, a more sophisticated approach is to handle the color channels in a "holistic" manner, using quaternions. We present a motion estimation approach which is based on the Hypercomplex (quaternionic) Phase Correlation of Moxey et al. We show that the quaternionic approach is computationally more efficient and more robust to sensor noise, compared with the corresponding three-channels-separately approach. With the assumption of smoothly time-varying velocities we propose the application of a weighted fuzzy c-means clustering procedure to the obtained velocity estimates. This renders the estimation more robust. A methodology in the hypercomplex Fourier transform domain for separating the moving objects/layers is also presented. An energy-minimization-based approach for the spatial assignment of the velocities and the creation of segmentation maps, is given. We furthermore show how to apply the motion estimation approach locally in space for the extraction of dense motion fields. Our experimental results and comparisons with state-of-the-art methodologies verify the effectiveness of the proposed approach. (C) 2008 Elsevier Inc. All rights reserved.