An Integrated Approach Combining Virtual Environments and Reinforcement Learning to Train Construction Robots for Conducting Tasks Under Uncertainties

Robots can support onsite workers with repetitive and physically demanding tasks (e.g., bricklaying) to reduce workers' risk of injuries. Central to the wide application of construction robots is solving the task of motion planning (i.e., moving objects optimally from one location to another under constraints such as joint angle limits). Currently, robots are mostly deployed in the manufacturing phase of a construction project for off-site production of building components. Motions of these robots are pre-programmed and follow strictly designed trajectories and actions. However, the motions of robots on construction sites require considerations of uncertainties, including the onsite movement of material and equipment, as well as changes to workpieces and target locations of the work piece. Therefore, it is essential to enable construction robots to handle these uncertainties while executing construction tasks to extend their applicability onsite. In this study, we proposed an integrated approach combining virtual environments and reinforcement learning (RL) to train robot control algorithms for construction tasks. We first created a virtual construction site using a game engine, which allows for the realistic simulation of robot movements. Next, the physical characteristics of the workpiece (e.g., location) were randomized in the virtual environment to simulate onsite uncertainties. An RL-based robot control algorithm (i.e., Proximal Policy Optimization) was implemented to train the robot for completing a construction task. We tested the robustness and effectiveness of the approach using a testbed construction site for window installation. Results showed that the proposed approach is effective in training the construction robot arm to handle window installation under the uncertainties of window location, with a success rate of 75% for picking up (i.e., grasping) the window and a success rate of 68% for placing the window to its target placement without crashing into other objects onsite. Researchers and practitioners can use the proposed approach to train control algorithms for their specific construction tasks to allow for flexible robot actions considering onsite uncertainties.