Image-Based Camera Localization for Large and Outdoor Environments

Locating camera position and orientation is an important step for many augmented reality (AR) applications. In this paper, we develop a system for estimating camera pose for large and outdoor environments. A large set of images for outdoor environments are collected and 3D structure of the scenes are recovered using a structure from motion technique. To improve image indexing accuracy and efficiency, a convolutional neural network (CNN) is employed to extract image features and a set of locality sensitive hashing (LSH) functions are used to classify CNN features. With these techniques, camera localization is achieved by first indexing the nearest images by CNN and LSH and then a set of 2D-3D correspondences are established from the indexed images and the recovered 3D structure. A perspective-n-point (PnP) algorithm is then applied on the 2D-3D correspondences to estimate camera pose. A series of experiments are conducted and the results confirm the effectiveness of proposed system. The nearest neighbors to query image can be accurately and efficiently extracted and the camera pose can be accurately estimated.