Deep learning-based approaches for unsupervised core-set selection

Abstract

Unsupervised coreset selection is a method of identifying samples with the most useful information given a pool of unlabeled data. Through the above technique, it is possible to reduce the human effort required to label each sample. In this thesis, we propose a method that enables contrast learning to be effectively applied to unsupervised core-set selection. The contrastive learning technique, a recently reported self-supervised method, is usually used to learn useful information from a data pool without labels. There are two leading methods for contrastive learning: a simple framework for contrastive learning of visual representations (SimCLR) and a momentum contrastive (MoCo) learning framework. For unsupervised core-set selection, we first proposed a core-set score, which is based on the accumulated cosine similarity calculated for every epoch while running the contrast learning from the unlabeled data pool. Our core set chosen by the core-set score exhibited an extremely good performance compared to the core-set chosen by random selection, and showed a similar performance as the existing supervised core set selection method. In addition, it was demonstrated theoretically and experimentally that the above method can be extremely useful for detecting redundant examples in an unsupervised manner (unsupervised redundancy identification). Unsupervised redundancy identification indicates the detection of a set of examples from a dataset that appear quite similarly to each other in the absence of an annotation.