Fast and robust distributed machine learning

Abstract

With the increasing number of edge nodes such as mobile phones, Internet of Things (IoT) devices and smart vehicles/drones, large amounts of valuable computing resources are now located at the edge. Moreover, a large portion of data generated nowadays is collected at these distributed nodes at the edge. In this sense, distributed machine learning (ML) is becoming significantly important and receiving considerable attention nowadays. This thesis proposes fast, robust and communication-computation efficient distributed machine learning (ML) solutions that are becoming increasingly important in current edge networks with a number of distributed nodes. In the first part of the thesis (Chapters 2 and 3), we focus on distributed learning which aims to speed up training via data parallelization. By utilizing the tools of information theory and coding theory, we develop a fast and robust distributed learning solution highly tailored to the practical communication networks. In the second part (Chapter 4), we focus on federated learning, where the goal is to collaboratively train a model without directly uploading the local/private data of each node to the server. We propose an algorithm that can speed up federated learning using multiple edge servers in a wireless setup. Finally in the third part (Chapter 5), we focus on split learning, which enables to reduce the computation burden at the clients by splitting the model architecture (i.e., the neural network) into the client-side model and the server-side model. We propose an algorithm that can address the latency and communication efficiency issues of current federated/split learning approaches, via a local-loss-based training method specifically geared to the split learning setup.