Recovering valuable information from large dataset

Abstract

Recovering valuable information from large datasets has become a major challenge in this era of data explosion. Large datasets seem to be random at first glance, but they are usually explainable through much small number of features. The goal of this thesis is to propose some computationally efficient, mathematical algorithms that recover the underlying signals or features. We theoretically analyze the algorithms under some probabilistic models, and we verify our findings with some simulations. Three different problems, binary classification with XOR queries, hypergraph clustering, matrix completion, are considered in this thesis. For the first problem, we provide a sharp threshold on the number of queries required to recover the binary labels, and we propose an algorithm based on belief propagation that achieves this. We assume a practical scenario where the error probability changes depending on the worker, and the proposed algorithm achieves the bound even without the knowledge of the worker reliability. For the hypergraph clustering, we first estimate the adjacency relation between nodes through convex optimization, and we recover the communities by clustering the rows of estimated adjacency matrix. Lastly, we studied nonconvex optimization and gradient descent in matrix completion. We prove that gradient descent converges to the global minima even in the presence of local minima or saddle points if accompanied with small random initialization.