3D Excavator Pose Estimation Using Projection-Based Pose Optimization for Contact-Driven Hazard Monitoring

Contact-driven accidents involving actuated excavators have led to a significant number of fatalities in the construction industry. The revolving mechanical arm of excavators poses a major risk of contact-driven accidents for workers in its proximity due to its articulated pose. Detecting the 3D pose of excavator arms is thus essential to prevent contact-driven accidents near excavators. Previous works have attempted to estimate 3D excavator poses using sensor-based or computer vision-based methods. However, existing methods require extensive preparation work, such as attaching physical sensors, calibrating stereo cameras, or collecting 3D training data. As a result, existing methods cannot be easily integrated into the current construction workflow and are seldom applied in real-world situations. The authors propose a projection-based 3D pose optimization method that utilizes excavator kinematic constraints to infer 3D excavator poses from monocular image sequences with no dependency on 3D training data. The proposed method first extracts the 2D excavator pose from images using a keypoint region-based convolution neural network. Then, the 2D pose is reconstructed into 3D by enforcing the rigid excavator kinematic constraints (e.g., arm length) and minimizing the 2D reprojection error of the excavator pose. Tests using a 1:14 miniature excavator model showed a 3D position error of 7.3 cm (or 1.03 m when scaled up to real-world dimensions) for keypoints on the excavator pose, demonstrating the capabilities of the proposed method in estimating 3D excavator poses from monocular images. The proximity measuring capacity of the estimated 3D pose was also evaluated, achieving a mean absolute distance error of 4.7 cm (or 0.66 m scaled). The proposed method offers a 3D excavator pose estimation method using only a monocular camera and without relying on 3D training data. The estimated 3D excavator pose enables safety managers to monitor potential contact-driven accidents near excavators and alert workers of unsafe situations and promotes safer working environments for construction workers near excavators.