Semi-automatic Motion Segmentation with Motion Layer Mosaics

A new method for motion segmentation based on reference motion layer mosaics is presented. We assume that the scene is composed of a set of layers whose motion is well described by parametric models. This usual assumption is compatible with the notion of motion layer mosaic, which allows a compact representation of the sequence with a small number of mosaics only. We segment the sequence using a reduced number of distant image-to-mosaic comparisons instead of a larger number of close image-to-image comparisons. Apart from computational advantage, another interest lies in the fact that motions estimated between distant images are more likely to be different from one region to another than when estimated between consecutive images. This helps the segmentation process. The segmentation is obtained by graph cut minimization of a cost function which includes an original image-to-mosaic data term. At the end of the segmentation process, it may happen that the obtained boundaries are not precisely the expected ones. Often the user has no other possibility than modifying manually every segmentation one after another or than starting over all again the process with different parameters. We propose an original easy way for the user to manually correct the possible errors on the mosaics themselves. These corrections are then propagated to all the images of the corresponding video interval thanks to a second segmentation pass. Experimental results demonstrate the potential of our approach.