A noise robust method for 2D shape estimation of moving objects in video sequences considering a moving camera

For the coding of image sequences at very low bit rates, it has been shown that the coding efficiency can be increased by taking into account the shape of moving objects in the scene. Moreover, the upcoming ISO/MPEG-4 standard considers the 2D shape of moving objects not only for reasons of coding efficiency, but also to provide the user with so-called content-based functionalities. However, the perfect automatic segmentation of moving objects in image sequences is still an unsolved problem. In this paper, an algorithm for automatic, noise robust 2D shape estimation of moving objects in video sequences is presented, which considers a moving camera. The algorithm consists of four main steps. In the first step, a possibly apparent camera motion is estimated and compensated. By the second step, a possibly apparent scene cut is detected, and if necessary the segmentation algorithm is reset. In the third step, a change detection mask is estimated by a relaxation technique, using local thresholds which consider the state of neighbouring pels. By the fourth step, regions where the background has been uncovered by a moving object are estimated, using motion information from a displacement vector field. Finally, the resulting object mask is adapted to luminance edges in the corresponding image, in order to improve the shape accuracy. Furthermore, the temporal coherency of the object mask is improved by applying a memory. The proposed algorithm is compared to an approach for 2D shape estimation of moving objects from the literature, which does not use any edge adaptation or object memory and which uses only one global threshold. The experimental results show that the resulting object shapes from the proposed algorithm look subjectively much better than those from the reference algorithm. (C) 1998 Elsevier Science B.V. All rights reserved.