Targetless Spatiotemporal Calibration of Multi-LiDAR Multi-IMU System Based on Continuous-Time Optimization

Nowadays, LiDAR-IMU systems have progressively prevailed in mobile robotic applications due to their excellent complementary characteristics. With the steady decline and shrinking in cost and size of these sensors, it has become feasible and even imperative to further leverage multiple sensor units for better accuracy and robustness. In such a fusion-based system, accurate spatiotemporal calibration is a fundamental prerequisite. However, existing calibration methods generally necessitate artificial targets or an overlapping field-of-view (FoV) between LiDARs. To this end, we propose an accurate and easy-to-use and spatiotemporal calibration approach tailored to multi-LiDAR multi-IMU systems based on continuous-time batch estimation, which supports both mechanical spinning LiDARs and small FoV solid-state LiDARs. Inspired by classical hand-eye calibration, a stepwise multistage nonlinear optimization problem is first built to recover the rotational extrinsic and poses of control points in the spline without using any artificial targets, special movements, or manual intervention. Meanwhile, a virtual IMU and LiDAR are constructed to bridge all IMUs and LiDARs based on the centralized principles in both temporal and spatial domains. Subsequently, the point-to-plane factors associated by different LiDARs, gyroscope, and accelerometer factors are jointly minimized to optimize all spatiotemporal parameters over multiple batches. Extensive simulation tests and real-world experiments were conducted to quantitatively evaluate the feasibility of the proposed method. Meanwhile, the proposed method is compared with the state-of-the-art methods and shows superior calibration performance.