Alphago for belief space planning

Abstract

In robotics, the ability to make decisions in an environment that includes uncertainty is essential. The robot should infer the current state through a series of actions and observations and plan to achieve the goal based on them. This problem can be modeled as Partially Observable Markov Decision Process (POMDP). However, high-dimensional state, action, and observation spaces in long horizon make a POMDP problem more complicated. A large number of simulations are needed to make a plan for this complex POMDP problem using previous online planning algorithms which do not guide a planner. In this paper, we learn policy network and value network by imitating prior experience data generated from simulations. Then use these networks to guide online planning algorithm to resolve complex POMDP problems more efficiently. We model the Light-Dark Room domain, one of localization problem in robotics, as a continuous POMDP. Our guided planning algorithm achieves higher success rates in this problem with less number of simulations.