Reinforcement learning control for a three-link biped robot with energy-efficient periodic gaits基于强化学习控制三连杆双足机器人实现节能周期步态

Designing a high-performance controller for the walking gaits of biped robots remains an open research area due to their strong nonlinearity and non-smooth responses. To overcome such challenges, a humanoid robot with a torso, i.e., a three-link biped robot involving both impact and friction, is developed firstly. Then, the twin delayed deep deterministic policy gradient algorithm is adopted to design the reinforcement learning controller for the proposed biped robot. For the specified control targets, i.e., energy-efficient periodic gaits for both the downhill and uphill cases, a reward function utilizing the Poincaré map and the power function is constructed to provide guidelines for the controller. Thus, the proposed controller can learn to adaptively output accurate cosine torques to achieve the goal without relying on the pre-designed reference trajectories or embedded unstable periodic gaits. A comparative study between the proposed reinforcement learning and neural network proportion differentiation controllers demonstrates the proposed controller can lead to accurate and energy-efficient periodic gaits and provide strong adaptability and robustness within a wide variety of walking slopes.