Vision-based Global Localization of Unmanned Aerial Vehicles with Street View Images

In this paper, a vision-based method is proposed to globally localize an unmanned aerial vehicle (UAV) flying in the urban scenarios using Google Street View as prior information. Since the trajectory of the UAV with absolute scale cannot be recovered with the help of monocular visual odometry only, we leverage the known positional information of Google Street View to localize vehicles with respect to the surrounding buildings. First, a place recognition algorithm is presented to solve air-ground matching problem by searching aerial images taken from onboard camera within the geotagged image database, Then based on the matched images by place recognition and tracked features via visual odometry, a novel absolute scale estimation mechanism is proposed to estimate the pose of UAV in the real world coordinate system and to correct the scale drift of visual odometry. Furthermore, the pose graph with UAV's poses as vertexes and relative transformations as edges is optimized with SE(3) constraints to get trajectory of UAV in the real world coordinate system. Experiments are conducted on a 2km dataset recorded onboard by a caniera-ectuipped UAV flying in the urban streets of Zurich, Switzerland, and the results show that our method is able to metrically localize the UAV without previously visiting or building 3D models in advance.