Towards robust neural networks and efficient exploration in reinforcement learning

Abstract

Deep neural networks have shown remarkable performance across a wide range of vision-based tasks, particularly due to the availability of large-scale datasets for training and better architectures. However, data seen in the real world are often affected by distortions that not accounted for by the training datasets. In the first part of this dissertation, we address the challenge of robustness and stability of neural networks and propose a general training method that can be used to make the existing neural network architectures more robust and stable to input visual perturbations while using only available datasets for training. The proposed training method is convenient to use as it does not require data augmentation or changes in the network architecture. We provide theoretical proof as well as empirical evidence for the efficiency of the proposed training method.In the second part of this dissertation, we study exploration in the context of reinforcement learn-ing. Deep reinforcement learning algorithms have been shown to learn complex skills using only high-dimensional observations and scalar reward. Effective and intelligent exploration still remains an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies such as $\epsilon-greedy$ exploration or adding Gaussian noise to actions. These heuristics, however,are unable to intelligently distinguish the well explored and the unexplored regions of the state space, which can lead to inefficient use of training time. We introduce entropy-based exploration (EBE) that enables an agent to explore efficiently the unexplored regions of the state space. EBE quantifies the agent's learning in a state using merely state dependent action values and adaptively explores the state space, i.e. more exploration for the unexplored region of the state space.In the end, we consider the problem of autonomous robotic navigation. We use deep Q-learning to train agents using both entropy-based exploration and the famous -greedy exploration heuristic to solve the task. We perform experiments under various environmental conditions and test the effectiveness of both of these exploration methods.