Vision-Aided Inertial Navigation System with Point and Vertical Line Observations for Land Vehicle Applications

Various aiding sensors can be integrated with the inertial navigation system (INS) to reduce its error growth when the vehicle is operating in GNSS denied environments. This paper developed a method to use the vanishing point from vertical line observations of building blocks in order to further improve point-based visual-inertial navigation system (VINS) for land vehicle applications. First, we presented the formulations of tightly coupled point-based VINS based on the Multi-State Constraint Kalman Filter (MSCKF) in the local-level frame. Second, we developed the relationship between the INS roll angle and vanishing point coordinates from vertical line observations. The roll angle measurement model is formulated. Finally, loosely coupled vertical line aiding module is added to the existing VINS, and the integration scheme is presented. Real world experiments demonstrated the validity of the mixed VINS method and the improved accuracy of the attitude and position estimation when compared with the solution without vertical line vanishing point aiding.