Deep introspective SLAM: deep reinforcement learning based approach to avoid tracking failure in visual SLAM

Abstract

Reliable and consistent tracking is essential to realize the dream of power-on-and-go autonomy in mobile robots. Our investigation with state-of-the-art visual navigation and mapping tools (e.g. ORB-SLAM) reveals that these tools suffer from frequent and unexpected tracking failures, especially when tested in the wild. This hinders the ability of robots to reach a goal position less than 10 meters away, without tracking failure, thereby limiting the prospects of real autonomy. We present an introspection-based approach (Introspective-SLAM) that enables SLAM to evaluate safety of navigation steps with respect to tracking failure, before the steps are actually taken. Navigation steps that appear unsafe are thereby avoided, and an alternative path to the goal is planned. We propose a novel deep reinforcement learning (DQN) based network to evaluate safety of future navigation steps using a single image only. Surprisingly, training of our DQN completes in a short amount of time (< 60 h). Even then, this network outperforms several handcrafted and Q-learning based pipelines to achieve state-of-the-art performance. Interestingly, training the DQN in realistic simulators (MINOS), consisting of reconstructed interiors, shows good generalization across real world indoor-outdoor settings. Finally, extensive testing of visual SLAM, equipped with our DQN, shows that tracking failures occur frequently and are a major hindrance in reaching the goal. Currently, there is no standard benchmark to evaluate active visual SLAM approaches. We have released a benchmark of 50 episodes with this work. We hope these findings/benchmark will encourage progress for power-on-and-go visual SLAM without any manual supervision.