Algorithm-architecture co-design for accelerating deep reinforcement learning on robotics applications

Abstract

Deep Reinforcement Learning (DRL) plays a critical role in controlling and manipulating futureintelligent machines like robots and drones. However, DRL repeats inference and training that are computationallyexpensive on resource-constraint mobile/embedded platforms. Even worse, DRL producesa severe hardware underutilization problem due to its unique execution pattern.This paper presents CoRe, an algorithm-architecture co-designed system that simultaneously achievesspeedup and energy efficiency of the DRL algorithm while maintaining its accuracy. To overcome theinefficiency of DRL, we propose Train Early Start, a new execution pattern for building the efficienton-policy DRL algorithm. Train Early Start parallelizes the inference and training execution, hidingthe serialized performance bottleneck. To maximize the benefit of Train Early Start, we design CoRearchitecture, a multi-core NPU system that exploits the task-level parallelism of Train Early Start. As a result, CoRe achieves on average 2.81× speedup and 6.27× energy efficiency over the state-of-the-artmobile SoC.