Smartphone-based Vision/MEMS-IMU/GNSS tightly coupled seamless positioning using factor graph optimization

With the diversification of functionalities and enhancement of performance in smartphones, low-cost GNSS chips, Micro-Electro-Mechanical System (MEMS) inertial measurement units (IMU), and cameras have become essential hardware components. Moreover, the computational power provided by smartphones for data processing has also become more robust. The fusion of multiple sensors in smartphones for positioning has emerged as a trend. In order to further enhance the positioning performance of smartphones in complex environments, this paper proposes a smartphone-based Vision/MEMS-IMU/GNSS tightly coupled integration for indoor-outdoor seamless positioning. Factor graph optimization is employed to combine observations from monocular vision, MEMS-IMU, and GNSS. To solve the unstable timing of current smartphone sensors, we present a GNSS-to-VIO time synchronization method. Furthermore, an adaptive positioning mechanism is devised based on the presence or absence of GNSS signals, allowing indoor and outdoor environment to switch. The proposed method is validated by integrating smartphone VIO with both smartphone built-in GNSS and high-performance geodetic GNSS receivers. The test results of the complex outdoor environments demonstrate that the proposed method significantly enhances GNSS positioning performance. The positioning accuracy of the proposed method using geodetic GNSS receiver is superior to that using smartphone GNSS. In outdoor environments, our method enables positioning accuracy of smartphone to improve 29.4, 53.0 and 69.9 % in three directions compared to single point positioning. Indoor and outdoor tests verified that the proposed method can provide continuous and reliable positioning in alternating interrupted and restored GNSS signals.