Three-dimensional reconstruction of environment using factorization method and EPI analysis

We propose a method for three-dimensional (3D) reconstruction of environment from image sequence taken with handy video camera. The convention EPI analysis is a popular method for 3D reconstruction from motion camera image sequence, but the camera needs to be moved linearly with constant velocity. We propose a method for enabling the EPI analysis to be applicable to image sequence taken with a handy video camera of unknown motion. The proposed method removes irregular shakings by hands from input image sequence by using the factorization method and tracking feature points. We apply the convention EPI analysis to the corrected image sequence as if which were taken by camera moved horizontally with constant velocity. The method provides 3D points of the feature points in the scene. In addition, we estimate surface of environmental structure from the 3D points obtained with the method. In the surface estimation, the 3D position of the feature points can be adjusted by matching the texture on the input images. Finally, the 3D model represented by triangle meshes is reconstructed. By rendering the corresponding image region to each mesh, the proposed method made it possible to reconstruct 3D scene with more reality than traditional one.