Autonomous Driving based on Modified SAC Algorithm through Imitation Learning Pretraining

Abstract

In this paper, we implement a modified SAC [1] algorithm for autonomous driving tasks using the simulator AirSim's [2] environment API which provides various weather, collision, and lighting choices. Given current image state and car velocity as our inputs, the task outputs the throttle, brake, and steering angle data and gives the vehicle action instruction through the AirSim control outputs. As autonomous vehicles are more likely to be accepted if they drive like how human would, we at first train our model by imitation learning to provides the pre-trained human-like policy and weights to SAC. During the reinforcement learning, in order to increase the feasible policy's robustness, we use ResNet-34 [3] as our actor and critic network architecture in the SAC algorithm.