Arbitrary viewpoint video synthesis from uncalibrated multiple cameras

In this paper, we propose a method for arbitrary view synthesis from uncalibrated multiple camera system, targeting large space such as soccer stadium. In Projective Grid Space (PGS), that is the three-dimensional space defined by epipolar geometry between the two basis cameras in the camera system, we reconstruct three dimensional shape models from the silhouette images. By using the three dimensional shape model reconstructed in PGS, we can obtain the dense point correspondence map between reference images. The obtained correspondence can synthesize the image of arbitrary view between the reference images. We also propose the method for merging the synthesized images with the virtual background scene in PGS. We apply the proposed method to image sequences taken by the multiple camera system, which is developed in a large space on a concert hall. The synthesized image sequences of virtual camera have enough quality to demonstrate effectiveness of the proposed method.