Linear sequence-to-sequence alignment

We present a novel approach for temporally aligning N unsynchronized sequences of a dynamic 3D scene, captured from distinct viewpoints. Unlike existing methods, which work for N = 2 and rely on a computationally-intensive search in the space of temporal alignments, we reduce the problem for general N to the robust estimation of a single line in RN. This line captures all temporal relations between the sequences and can be computed without any prior knowledge of these relations. Experimental results show that our method can accurately align sequences even when they have large mis-alignments (e.g., hundreds of frames), when the problem is seemingly ambiguous (e.g., scenes with roughly periodic motion) and when accurate manual alignment is difficult (e.g., due to slow-moving objects).