Energy-efficient deep-neural-network training processor with fine-grained mixed precision

Abstract

Recently, several hardware accelerators have been reported for deep neural network (DNN) operation, however, they focused on only inference rather than DNN training that is a crucial ingredient for user adaptation at the edge-device as well as transfer learning with domain-specific data. However, DNN training requires much heavier floating-point (FP) computation and memory access than DNN inference, thus, dedicated DNN training hardware is essential. In this dissertation, we present a deep learning neural processing unit (LNPU) supporting CNN and FC training as well as inference with the following key features. First, we proposed fine-grained mixed precision (FGMP) scheme. The FGMP divides data into FP8-group and FP16-group in data-element level. FGMP can dynamically adjust the ratio between FP8 and FP16 to reduce external memory access and avoid accuracy loss. With the FGMP, external memory access is reduced by 38.9% for ResNet-18 training. Second, we designed hardware architecture to support FGMP. For high energy efficiency, we proposed DL core architecture with configurable PE and data-path for DNN training with FGMP. As a result, the energy efficiency of LNPU is improved by $2.08 \times$ ResNet-18 training. Lastly, we proposed fully-reconfigurable hardware architecture for various kinds of operations in DNN training/inference with zero-skipping. With the help of fully-reconfigurable hardware architecture, proposed LNPU can support all of the steps of DNN training with skipping zeros which are derived from FGMP and ReLU, and so on. As a result, the energy efficiency is $ \times 4.4$ higher than NVIDIA V100 GPU and its normalized peak performance is $\times 2.4$ higher than the previous DNN training processor.