(An) energy-efficient multiple-DNN training processor with active input-output dual zero skipping

Abstract

Recently, many deep neural network (DNN) based services are moving towards the edge for on-device intelligence, which led to the growth of research in energy-efficient DNN accelerators. Furthermore, as the functionality of artificial intelligence (AI) advances, their architectures become more complex, incorporating multiple DNNs in a single AI model. A generative adversarial network (GAN) is an example that can perform advanced applications through its multiple-DNN architecture. This dissertation presents GANPU, an energy-efficient multiple deep neural network (DNN) training processor for GANs. It enables on-device training of GANs on performance-limited and battery-limited mobile devices, without sending user-specific data to servers, fully evading privacy concerns. Training GANs require a massive amount of computation, therefore it is difficult to accelerate in a resource-constrained platform. Besides, networks and layers in GANs show dramatically changing operational characteristics, making it difficult to optimize the processor's core and bandwidth allocation. For higher throughput and energy-efficiency, this paper proposed 3 key features. An adaptive spatio-temporal workload multiplexing is proposed to maintain high utilization in accelerating multiple DNNs in a single GAN model. To take advantage of ReLU sparsity during both inference and training, dual-sparsity exploitation architecture is proposed to skip redundant computations due to input and output feature zeros. Moreover, an exponent-only ReLU speculation algorithm is proposed along with its light-weight processing element architecture, to estimate the location of output feature zeros during the inference with minimal hardware overhead. Fabricated in a 65 nm process, the GANPU achieved the energy-efficiency of 75.68 TFLOPS/W for 16-bit floating-point computation, which is 4.85x higher than the state-of-the-art. As a result, GANPU enables on-device training of GANs with high energy-efficiency.