(An) energy-effcient deep neural network acceleration by exploiting cross-layer weight scaling and bit-level data sharing

Abstract

Various accelerators have been proposed to efficiently execute deep neural networks in mobile devices with severe energy and area limitations. However, these accelerators mostly use fixed-point representations for energy-efficient inferencing and are optimized for higher-level vision tasks such as classification and recognition. In this dissertation, overcoming these limitations to perform various tasks energy-efficiently with minimal overhead in existing accelerators is proposed. To this end, when performing diverse tasks of the deep neural network, two methods are used to minimize access to external memory, which is the primary energy consumption source of the accelerator. A brief introduction to both ways follows.

1. Reformation of deep neural networks considering quantization error: Most deep neural networks trained and distributed by suppliers use floating-point representation for high prediction accuracy. However, embedded accelerators use a fixed-point representation for energy-efficient operation. Therefore, a quantization process that converts to a low-precision fixed-point representation while maintaining high accuracy is indispensable. In this dissertation, deep neural network modification methods are proposed to make robust from quantization errors by predicting and analyzing errors in the quantization process.

2. Data compression using spatial correlation: To reduce the amount of external memory access data used by the built-in accelerator, most of them use compression methods that take advantage of the activation function's sparsity to support parallel operations. However, depending on the deep neural network and the type of data, there are cases in which sufficient sparsity to use the existing compression method cannot be provided. This dissertation proposes compression methods that can effectively reduce communication even for deep neural networks and data possing low sparsity by using spatial correlation.

In summary, this dissertation proposes energy-efficient executing strategies for performing various tasks on embedded accelerators. The comprehensive goal is to improve the deep neural-network accelerator's energy efficiency. The main goal is to effectively reduce the amount of communication with external memory during execution with negligible overhead. To this subject, the data characteristics of deep neural networks and the limitations of accelerators are analyzed, and the proposed methods solve the raised problem.