DC Field | Value | Language |
---|---|---|
dc.contributor.author | Lee, Jung Hyun | ko |
dc.contributor.author | Yun, Jihun | ko |
dc.contributor.author | Hwang, Sung Ju | ko |
dc.contributor.author | Yang, Eunho | ko |
dc.date.accessioned | 2021-12-14T06:51:35Z | - |
dc.date.available | 2021-12-14T06:51:35Z | - |
dc.date.created | 2021-12-03 | - |
dc.date.created | 2021-12-03 | - |
dc.date.created | 2021-12-03 | - |
dc.date.created | 2021-12-03 | - |
dc.date.issued | 2021-10-14 | - |
dc.identifier.citation | 18th IEEE/CVF International Conference on Computer Vision (ICCV), pp.5370 - 5379 | - |
dc.identifier.uri | http://hdl.handle.net/10203/290618 | - |
dc.description.abstract | Network quantization, which aims to reduce the bit-lengths of the network weights and activations, has emerged for their deployments to resource-limited devices. Although recent studies have successfully discretized a full-precision network, they still incur large quantization errors after training, thus giving rise to a significant performance gap between a full-precision network and its quantized counterpart. In this work, we propose a novel quantization method for neural networks, Cluster-Promoting Quantization (CPQ) that finds the optimal quantization grids while naturally encouraging the underlying full-precision weights to gather around those quantization grids cohesively during training. This property of CPQ is thanks to our two main ingredients that enable differentiable quantization: i) the use of the categorical distribution designed by a specific probabilistic parametrization in the forward pass and ii) our proposed multi-class straight-through estimator (STE) in the backward pass. Since our second component, multi-class STE, is intrinsically biased, we additionally propose a new bit-drop technique, DropBits, that revises the standard dropout regularization to randomly drop bits instead of neurons. As a natural extension of DropBits, we further introduce the way of learning heterogeneous quantization levels to find proper bit-length for each layer by imposing an additional regularization on DropBits. We experimentally validate our method on various benchmark datasets and network architectures, and also support a new hypothesis for quantization: learning heterogeneous quantization levels outperforms the case using the same but fixed quantization levels from scratch. | - |
dc.language | English | - |
dc.publisher | Computer Vision Foundation, IEEE Computer Society | - |
dc.title | Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss | - |
dc.type | Conference | - |
dc.identifier.wosid | 000797698905057 | - |
dc.identifier.scopusid | 2-s2.0-85127806375 | - |
dc.type.rims | CONF | - |
dc.citation.beginningpage | 5370 | - |
dc.citation.endingpage | 5379 | - |
dc.citation.publicationname | 18th IEEE/CVF International Conference on Computer Vision (ICCV) | - |
dc.identifier.conferencecountry | CN | - |
dc.identifier.conferencelocation | Virtual | - |
dc.identifier.doi | 10.1109/ICCV48922.2021.00532 | - |
dc.contributor.localauthor | Hwang, Sung Ju | - |
dc.contributor.localauthor | Yang, Eunho | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.