Unsupervised segmentation of highly dynamic scenes through global optimization of multiscale cues

We propose a novel method for highly dynamic scene segmentation by formulating foreground object extraction as a global optimization framework that integrates a set of multiscale spatio-temporal cues. The multiscale features consist of a combination of motion and spectral components at a pixel level as well as spatio-temporal consistency constraints between superpixels. To compensate for the ambiguities of foreground hypothesis due to highly dynamic and cluttered backgrounds, we formulate salient foreground mapping as a convex optimization of weighted total variation energy, which is efficiently solved by using an alternating minimization scheme. Moreover, the appearance and position spatio-temporal consistency constraints between superpixels are explicitly incorporated into a Markov random field energy functional for further refinement of the set of salient pixel-level foreground mapping. This work facilitates sequential integration of multiscale probability constraints into a global optimal segmentation framework to help address object boundary ambiguities in the case of highly dynamic scenes. Extensive experiments on challenging dynamic scene data sets demonstrate the feasibility and superiority of the proposed segmentation approach. (C) 2015 Elsevier Ltd. All rights reserved.