Free-viewpoint Image Generation from a Video Captured by a Handheld Camera

In general, free-viewpoint image is generated by captured images by a camera array aligned on a straight line or circle. A camera array is able to capture synchronized dynamic scene. However, camera array is expensive and requires great care to be aligned exactly. In contrast to camera array, a handy camera is easily available and can capture a static scene easily. We propose a method that generates free-viewpoint images from a video captured by a handheld camera in a static scene. To generate free-viewpoint images, view images from several viewpoints and information of camera pose/positions of these viewpoints are needed. In a previous work, a checkerboard pattern has to be captured in every frame to calculate these parameters. And in another work, a pseudo perspective projection is assumed to estimate parameters. This assumption limits a camera movement. However, in this paper, we can calculate these parameters by "Structure From Motion". Additionally, we propose a selection method for reference images from many captured frames. And we propose a method that uses projective block matching and graph-cuts algorithm with reconstructed feature points to estimate a depth map of a virtual viewpoint.