DC Field | Value | Language |
---|---|---|
dc.contributor.author | Shin, Jae Woong | ko |
dc.contributor.author | Lee, Hae Beom | ko |
dc.contributor.author | Gong, Boqing | ko |
dc.contributor.author | Hwang, Sung Ju | ko |
dc.date.accessioned | 2021-12-10T06:49:16Z | - |
dc.date.available | 2021-12-10T06:49:16Z | - |
dc.date.created | 2021-11-30 | - |
dc.date.created | 2021-11-30 | - |
dc.date.created | 2021-11-30 | - |
dc.date.issued | 2021-07-18 | - |
dc.identifier.citation | 38th International Conference on Machine Learning, ICML 2021 | - |
dc.identifier.issn | 2640-3498 | - |
dc.identifier.uri | http://hdl.handle.net/10203/290420 | - |
dc.description.abstract | Meta-learning of shared initialization parameters has shown to be highly effective in solving few-shot learning tasks. However, extending the framework to many-shot scenarios, which may further enhance its practicality, has been relatively overlooked due to the technical difficulties of meta-learning over long chains of inner-gradient steps. In this paper, we first show that allowing the meta-learners to take a larger number of inner gradient steps better captures the structure of heterogeneous and large-scale task distributions, thus results in obtaining better initialization points. Further, in order to increase the frequency of meta-updates even with the excessively long inner-optimization trajectories, we propose to estimate the required shift of the task-specific parameters with respect to the change of the initialization parameters. By doing so, we can arbitrarily increase the frequency of meta-updates and thus greatly improve the meta-level convergence as well as the quality of the learned initializations. We validate our method on a heterogeneous set of large-scale tasks, and show that the algorithm largely outperforms the previous first-order meta-learning methods in terms of both generalization performance and convergence, as well as multi-task learning and fine-tuning baselines. | - |
dc.language | English | - |
dc.publisher | International Machine Learning Society | - |
dc.title | Large-Scale Meta-Learning with Continual Trajectory Shifting | - |
dc.type | Conference | - |
dc.identifier.wosid | 000768182705069 | - |
dc.identifier.scopusid | 2-s2.0-85161272554 | - |
dc.type.rims | CONF | - |
dc.citation.publicationname | 38th International Conference on Machine Learning, ICML 2021 | - |
dc.identifier.conferencecountry | US | - |
dc.identifier.conferencelocation | Virtual | - |
dc.contributor.localauthor | Hwang, Sung Ju | - |
dc.contributor.nonIdAuthor | Shin, Jae Woong | - |
dc.contributor.nonIdAuthor | Gong, Boqing | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.