Real-Time Building Instance Recognition for Vector Map and Real Scene Fusion

Objectives: In current navigation systems, the vector map navigation lacks real environment information while visual geolocation rely heavily on massive image annotation data, leading to unsatisfactory experience for general users. This paper proposes a method of real-time recognition and positioning of single buildings for mobile augment reality (AR) navigation. Methods: The proposed method adopts a lightweight deep network SSD (single shot detector) to detect in real-time the building objects from the mobile phone video stream, and obtains the current position and shooting angle of view by using the built-in sensors of the mobile phone. Once the building category is recognized, the attributes and positioning information of the involved building instances are able to be obtained by exploiting the vector map information, which are superposed on the vector map to be visualized. Thereby, an enhanced navigation system combining real geographic environment and vector map is achieved. Results: The experimental results show that our proposed method can correctly identify multiple building entities at different times and locations. The building detection is less affected by lighting conditions, and the detection accuracy can reach about 95%, which meets the requirements of real-time navigation. Conclusions: Compared with the traditional geolocation method, this method can make full use of the complementary information of vector maps and realistic photos, and only requires a small number of building annotation samples. This proposed method succeeds in realizing mobile AR navigation with enhanced information of individual buildings, which effectively relieves the unintuitive visualization problem of vector map navigation. This study can potentially improve users' experience and cognitive ability of environment through building detection and information enhancement. © 2023 Wuhan University. All rights reserved.