(A) redundancy-aware architecture to eliminate repeated computations in quantized convolutional neural networks

Abstract

Quantized Convolutional Neural Networks (QCNNs) have been proposed to reduce the memory size that is one of the major bottleneck of CNNs. Nevertheless, the massive number of computations still hinders an efficient processing of deep CNNs in modern devices. To reduce to computations while keeping the reduced memory size, this work is motivated by following two observations. First, there are a large number of redundant computations due to the repeated operands caused by quantization. The other is that QCNNs have extremely low or no sparsity in weights. Most of the previous accelerators have only considered the redundant computations caused by sparsity, not the repeated data, resulting in less improvement of performance in QCNNs compared to the high precision CNNs because of the lack of sparsity. This paper introduces RQ-CNN, a novel accelerator architecture that eliminates all the redundant computations in QCNNs. RQ-CNN identifies the indispensable data referred to both unique and non-zero data in the network. Through the identification process, the redundant computations are processed at one time by a merged multiplier using the indispensable data. While exploiting the redundancy, RQ-CNN maintains the parallelism and reusability of CNNs by fixing the redundancy scope that is a window for searching indispensable data without messing the data. On the state-of-the-art QCNNs, RQ-CNN improves the performance and energy by a factor of $4\times$ and $2.19\times$, respectively.