Machine learning methods for black-box optimization and image classification

Abstract

This dissertation discusses the development of Bayesian optimization, which is one of black-box optimization methods, and the development of meta-learning and capsule networks, a deep learning technique that performs image classification. More detail, Bayesian optimization and meta-learning assume that only a few amount of data can be observed, and in this situation, inaccurate inference are induced by suffering from the lack of information. This dissertation focuses on addressing this difficulty caused by the lack of data. In the study of Bayesian optimization, we propose a model that incorporates the prior knowledge about the bound of an optimal value, and show the improvement in Bayesian optimization with the proposed model. In short, the difficulty caused by the lack of information is relaxed by adapting the prior information to the model. In the study of meta-learning, we develop the algorithm that performs out-of-distribution (OOD) detection and classification simultaneously. In fact, meta-learning has already proposed a way to overcome the difficulty caused by the lack of training data, but it has not provided a solution for learning about OOD. The study of meta-learning in this dissertation contributes to proposing a framework for learning OOD when only few shots of data are available. In the study of capsule networks, we focus on addressing the difficulty of training in capsule network. In this study, we first introduce pruning layer and ladder layer and finally propose ladder capsule network, which is an alternative of the previous capsule network. From the pruning and ladder layer, ladder capsule network can reduce the unnecessary computation in the previous capsule network and shows similar performances as the capsule network.