Towards real-world meta-learning

Abstract

We extend the conventional meta-learning frameworks to more realistic, practical, and large-scale learning scenarios. Firstly, realistic meta-learning assumes imbalances between classes and tasks, and also distributional shift between meta-training and meta-testing stage. Secondly, practical meta-learning aims to develop a versatile meta-knowledge that is agnostic to architectural differences. Lastly, we address large-scale meta-learning where a shared initialization or hyperparameter are efficiently learned over a heterogeneous set of many-shot tasks. In this paper, we show how we can efficiently and effectively address those challenging real-world meta-learning problems with various machine learning techniques such as variational inference, amortization, first-order approximation, Taylor approximation, Lipschitz assumption, and so on.