Localization method by aerial image matching in urban environment based on semantic segmentation

To address the problem that how to obtain the position of unmanned aerial vehicles (UAVs) in global navigation satellite system (GNSS)-denied environment or weak GNSS environment to enable them accomplish various established tasks, a method to obtain the absolute position of UAV in urban environment was proposed, by using a downward-facing monocular camera and an altimeter on the UAV as sensors, and the pre-existing reference satellite imagery to align with aerial imagery．Before alignment, the convolutional neural network (CNN) was used to extract the building information of the aerial imagery and pre-existing reference satellite imagery through semantic segmentation． In contrast to the traditional point feature matching method, by utilizing the building arrangement information to align the two imageries, and therefore the influence of the difference between the two imageries could be eliminated, such as season changes and so on．The proposed method could be performed without knowing the exact initial position of UAV, and the initialization algorithm could be used to estimate the initial position of UAV in the range of a large area．The iterative closest point (ICP) algorithm was used to eliminate the drift in the visual odometry (VO) process．Finally, experiment on the dataset simulated in Google Earth verified the effectiveness of the proposed method. © 2022 Huazhong University of Science and Technology. All rights reserved.