DC Field | Value | Language |
---|---|---|
dc.contributor.author | Zandieh, Amir | ko |
dc.contributor.author | Han, Insu | ko |
dc.contributor.author | Avron, Haim | ko |
dc.contributor.author | Shoham, Neta | ko |
dc.contributor.author | Kim, Chaewon | ko |
dc.contributor.author | Shin, Jinwoo | ko |
dc.date.accessioned | 2021-12-09T06:47:59Z | - |
dc.date.available | 2021-12-09T06:47:59Z | - |
dc.date.created | 2021-12-02 | - |
dc.date.issued | 2021-12-07 | - |
dc.identifier.citation | 35th Conference on Neural Information Processing Systems, NeurIPS 2021 | - |
dc.identifier.uri | http://hdl.handle.net/10203/290291 | - |
dc.description.abstract | The Neural Tangent Kernel (NTK) characterizes the behavior of infinitely-wide neural networks trained under least squares loss by gradient descent. Recent works also report that NTK regression can outperform finitely-wide neural networks trained on small-scale datasets. However, the computational complexity of kernel methods has limited its use in large-scale learning tasks. To accelerate learning with NTK, we design a near input-sparsity time approximation algorithm for NTK, by sketching the polynomial expansions of arc-cosine kernels: our sketch for the convolutional counterpart of NTK (CNTK) can transform any image using a linear runtime in the number of pixels. Furthermore, we prove a spectral approximation guarantee for the NTK matrix, by combining random features (based on leverage score sampling) of the arc-cosine kernels with a sketching algorithm. We benchmark our methods on various large-scale regression and classification tasks and show that a linear regressor trained on our CNTK features matches the accuracy of exact CNTK on CIFAR-10 dataset while achieving 150x speedup. | - |
dc.language | English | - |
dc.publisher | Neural Information Processing Systems | - |
dc.title | Scaling Neural Tangent Kernels via Sketching and Random Features | - |
dc.type | Conference | - |
dc.type.rims | CONF | - |
dc.citation.publicationname | 35th Conference on Neural Information Processing Systems, NeurIPS 2021 | - |
dc.identifier.conferencecountry | US | - |
dc.identifier.conferencelocation | Virtual | - |
dc.contributor.localauthor | Shin, Jinwoo | - |
dc.contributor.nonIdAuthor | Zandieh, Amir | - |
dc.contributor.nonIdAuthor | Han, Insu | - |
dc.contributor.nonIdAuthor | Avron, Haim | - |
dc.contributor.nonIdAuthor | Shoham, Neta | - |
dc.contributor.nonIdAuthor | Kim, Chaewon | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.