Interactive collision detection for free-viewpoint video

We present a novel way of interacting with a virtual 3D scene in the context of free-viewpoint video. Using a multi-camera setup, our technique detects collisions between virtual objects and real objects, including people. We perform collision computations directly on the image data, as opposed to reconstructing the full geometry of the subject. This reduces implementation complexity, and moreover, yields interactive performance. We demonstrate the effectiveness of our technique by incorporating it in a rigid body simulation. The subject can interact with virtual objects and observe his or her actions while being able to adjust the viewpoint, all in real-time.