Quadruped locomotion on non-rigid terrain using reinforcement learning

Abstract

Generating quadruped locomotion using reinforcement learning has shown promising results so far. Imitation learning was used to create diverse locomotion styles. Methods based on learning without data showed improved results in speed and accuracy on following commands than methods using a manually designed controller. Indeed, quadruped locomotion both in computer graphics and robotics are becoming more robust. Despite these progresses, most current works solved locomotion on rigid terrain. Since we can easily imagine non-rigid terrain in a real environment such as sand or mud, locomotion on non-rigid terrain should be solved. Our work aims to tackle locomotion on non-rigid terrain. We first showed the effectiveness of our method on the flat rigid terrain. Then we extend our method to work in non-rigid terrain. Our method is based on creating three Bezier curves for the current moving end-effector, the base position, and the base orientation respectively. Once the Bezier curve is created through action space, inverse kinematics is used to calculate the joint angles needed to reach that point. We chose control points of the Bezier curve as an action space. Curriculum learning that enlarged the freedom of control point position enhanced the performance by producing longer stable locomotion. In observation space, we included the configuration of the robot and the minimum amount of environment which was induced by the sparse ray projected from the robot. The necessity of each reward is proven by showing the result motion without each of the rewards subtracted.