Real-time moving object detection and segmentation in H.264 video streams

In this paper we present a novel algorithm for moving object detection and segmentation, operating on H.264 bit streams. Compared to more traditional pixel-based approaches, the novelty of the algorithm consists of directly using the motion features embedded into the H.264 bit stream, thereby achieving real time operational capability. This makes the algorithm ready to be installed in any video surveillance system, enabling for better resource allocation and facilitating the deployment of distributed systems. The method we propose measures the statistical disorder of the motion field at the boundary of the moving objects, achieving at the same time detection and segmentation. In order to refine the segmentation, results, the temporal correlation of motion vectors is analyzed. The algorithm has been tested on the traditional videos used to benchmark video compression algorithms, as well as on a subset of sequences from the iLids dataset, to demonstrate its generalization capabilities.