Online Video-based Sequence Synchronization for Moving Camera Object Detection

This work presents a video-based sequence synchronization algorithm to be used in real-time video surveillance applications. The signals are aligned based on an online dynamic time warping approach that uses only video content information. The algorithm was tested in the alignment of reference and target videos acquired in a cluttered industrial environment with a moving camera. During each recording, the camera follows approximately the same trajectory but its speed along the trajectory may differ among recordings, thus generating time warping between the videos. The results shows that, when considering both alignment error and computational complexity, the use of the mean squared error as similarity metric leads to the best performance among metrics such as the SSIM, absolute error or even sophisticated matching techniques such as histograms of oriented gradients. The proposed method was also tested in a real-time application. Results show that the instantaneous alignment computation of a target frame can be performed successfully, at the cost of a three-fold increase in the error when compared to the case when latency is allowed.