Rapid Speed Change for Quadruped Robots via Deep Reinforcement Learning

This paper proposes a rapid speed change method during galloping through reinforcement learning designed to imitate the motion of quadrupedal locomotion. For the stable galloping motion of the quadrupedal robot, the force between the ground and the foot, that is, the desired ground reaction force (GRF), is calculated based on the linear momentum equation. When training the policy, the reward function is designed to give a high reward when the vertical force of the robot foot is similar to the desired GRF. The robot foot motion learned with this policy is simple, efficient, and similar to the galloping motion of a four-legged animals. Reward function was designed by observing the characteristics of the four-legged animal's body and foot motion during acceleration and deceleration. Since these motions of animals are innately learned to move most efficiently and stably, the robot was configured to control the robot by imitating these motions. When accelerating, four-legged animals use the large muscles of the rear limbs to make a strong pushing motion, so the reward function was designed to generate a larger desired GRF on the rear feet than on the front feet. Computer simulation were done based on the Unitree Go1 robot model using the NVIDIA Issac gym simulator, and results showed that the proposed method is effective for fast speed change through hardware tests using sim-to-real transfer.