Adaptive color background modeling for real-time segmentation of video streams

We present a system to perform real-time background modeling and segmentation of video streams on a PC, in the context of video surveillance and multimedia applications. The images, captured with a fixed camera, are modeled as a fixed or slowly changing background, which may become occluded by mobile agents. The system learns a statistical color model of the background, which is used for detecting changes produced by occluding elements. We propose to operate in the Hue-Saturation-Value (HSV) color space, instead of the traditional RGB space, and show that it provides a better use of the color information, and naturally incorporates gray-level only processing. At each instant, the system maintains an updated background model, and a list of occluding regions that can then be tracked. Other applications are video compression, enhancement and modification, such as obstacle highlight or removal.