Targetless Spatiotemporal Calibration for Multiple Heterogeneous Cameras and IMUs Based on Continuous-Time Trajectory Estimation

Nowadays, visual-inertial systems have been extensively studied and applied in mobile robots and autonomous vehicles due to their small footprint, low cost, and excellent complementary characteristics. To further the robustness and accuracy of such systems, integrating multiple cameras and inertial measurement units (IMUs) is a practically effective and commercially affordable choice. For such multisensor systems, accurate and consistent spatiotemporal calibration is a fundamental prerequisite. However, existing calibration methods generally require auxiliary artificial targets, as well as overlapping field of views between cameras, which significantly limits their usability. To this end, we propose a targetless spatiotemporal calibration method for multicamera multi-IMU systems, which supports both global shutter (GS) and rolling shutter (RS) cameras and enables intrinsic refinement. In particular, a novel continuous-time-based preintegration initialization approach is developed for calibration initialization, which can easily be extended to estimator initialization of IMU-integrated multisensor systems. Following the initialization procedure, several batch optimizations are performed, where all spatiotemporal parameters can be optimized in a consistent estimator by introducing a virtual central IMU. Both simulation tests and real-world experiments were carried out for quantitative evaluation. The results demonstrate that the proposed method is capable of accurate and consistent spatiotemporal calibration and intrinsic refinement and outperforms other state-of-the-art methods.