Accuracy-aware efficient neural network compression for resource-limited system

Abstract

Network compression algorithm has been used to accelerates the processing of deep neural networks by reducing the number of trainable parameters. In SVD-based network compression, the rank of decomposed convolution filter is a key parameter that determines the complexity and accuracy of a neural network. In this paper, we first introduce the combinatorial rank configuration algorithm with SVD-based filter pruning. The proposed approaches choose a rank configuration satisfying the constraint of computational complexity or accuracy in the combinatorial search space, while the previous works iteratively determine the rank for a layer. To implement the feasible combinatorial space, we propose the bounded space generation algorithms to extract the essential rank configurations among the numerous combinations. Also, the novel accuracy metrics are proposed to represent the accuracy and complexity relationship for a given neural network. We use these metrics to quickly evaluate the accuracy of rank configuration. Finally, we propose a single-shot approach to choose a rank configuration in a few seconds. Experiments show that our method provides better compromise between accuracy and computational complexity/memory consumption while performing compression at much higher speed. For VGG-16 our network can reduce the FLOPs by 25% and improve accuracy by 0.7% compared to the baseline, while requiring only 3 minutes on a CPU to search for the right rank configuration. Previously, similar results were achieved in 4 hours with 8 GPUs. The better accuracy and complexity compromise, as well as the extremely fast speed of our method makes it suitable for neural network compression. We expect that our approaches can be widely used for the acceleration of high performance algorithm using many neural networks and the real-time resource management system for the resource-limited applications such as mobile and robotics.