Quadrupedal Locomotion in an Energy-efficient Way Based on Reinforcement Learning

Achieving energy-efficient motion is important for the application of quadruped robots in a wide range. In this paper, we propose a hierarchical control framework that combines reinforcement learning and virtual model control to achieve energy-efficient motion with a planned gait. A reinforcement learning network is designed to learn the policy that maps the state of the robot to the action. The action is the increment of stance ratio, one of the gait parameters. The learned policy network is used as a high-level gait parameter modulator to adjust the gait parameters according to the body's velocity. The virtual model control method is used to compute the required force of robot's body. Then this force is decomposed to the feet of the stance legs with quadratic programming optimization. In the lowest level, the proportional-derivative controllers are used to control the joints' motion. Simulation and experiments are well conducted on the robot A1. The experimental results verify the effectiveness of the proposed method.