Hierarchical Multicontact Motion Planning of Hexapod Robots With Incremental Reinforcement Learning

Legged locomotion in unstructured environments with static and dynamic obstacles is challenging. This article proposes a novel hierarchical multicontact motion planning method with incremental reinforcement learning (HMC-IRL) that enables hexapod robots to pass through large-scale discrete complex unstructured environments with local changes occurring. First, a novel hierarchical structure and an information fusion mechanism are developed to decompose multicontact motion planning into two stages: planning the high level prior grid path and planning the low level detailed center of mass (COM) and foothold sequences based on the prior grid path. Second, we leverage the HMC-IRL method with an incremental architecture to enable swift adaptation to local changes in the environment, which includes incremental soft Q-learning (ISQL) algorithm to obtain the optimal prior grid path and incremental proximal policy optimization (IPPO) algorithm to obtain the COM and foothold sequences in the dynamic plum blossom pile environment. Finally, the integrated HMC-IRL method is tested on both simulated and real systems. All the experimental results demonstrate the feasibility and efficiency of the proposed method. Videos are shown at http://www.hexapod.cn/hmcirl.html.