Tightly-coupled GNSS/INS/Vision integration with semantic information via hybrid extended-unscented Kalman filtering

In recent years, integrating global navigation satellite system (GNSS) and visual-inertial odometry has become increasingly prevalent for intelligent system navigation. However, filter-based approaches face challenges of severe linearization errors, while optimization-based methods are disturbed by heavy computational burdens. Furthermore, the semantic information implying visual feature reliability is generally neglected for navigation usage. To address such issues, we propose a visual grading processing strategy based on semantic segmentation, classifying features by reliability and updating them through differential routines. Meanwhile, for the balance of accuracy and efficiency, we establish an extended-unscented-hybrid filtering framework that considers model nonlinearities, to tightly couple GNSS pseudorange and carrier-phase observations, inertial measurement unit records, visual stable and static features. The vehicular experiments demonstrate that the proposed system could obtain submeter-level positioning accuracy, outperforming the state-of-the-art filter-based algorithm and open- sourced optimization-based VINS-Fusion. Moreover, the efficiency assessment indicates that the proposed system can achieve real-time processing on a laptop.