3D live: Real time captured content for mixed reality

We present a complete system for live capture of 3-D content and simultaneous presentation in augmented reality. The user sees the real world. m his viewpoint, but fro modified so that the image of a remote collaborator is rendered into the scene. Fifteen cameras surround the collaborator, and the resulting video streams are used to construct a three-dimensional model of the subject using a shape-from-silhouette algorithm. Users view a two-dimensional fiducial marker using a video-see-through augmented reality interface. The geometric relationship between the marker and head-mounted camera is calculated, and the equivalent view of the subject is computed and drawn into the scene. Our system can generate 384 x 288 pixel images of the models at 25fps, with a latency of <100ms. The result gives the strong impression that the subject is a real part of the 3-D scene. We demonstrate applications of this system in 3-D video-conferencing and entertainment.