Learning quadrupedal locomotion on deformable terrain

Abstract

Simulation-based reinforcement learning approaches are leading the next innovations in legged robot control. However, the resulting control policies are still not applicable on soft and deformable terrains, especially at high speed. The primary reason is that reinforcement learning approaches, in general, are not effective beyond the data distribution: the agent cannot perform well in environments that it has not experienced. To this end, we introduce an accurate and computationally efficient granular media model for reinforcement learning. Our model can be parameterized to represent diverse types of terrain from soft beach sand to hard asphalt. In addition, we introduce an adaptive control architecture which can identify the terrain properties as the agent runs over the terrain. The identified parameters are then used to boost the locomotion performance of the legged robot. We applied our new methods to the Raibo robot, a dynamic quadrupedal robot developed in-house. The trained networks demonstrated unprecedented locomotion capabilities: the robot was able to run on beach sand at 3.03 m/s even though the feet are completely buried into the sand during the stance phase. The same policy was able to make the robot run on wet sand, soil, soft air mattress, and hard asphalt.