Solving Factored MDPs with Large Action Space Using Algebraic Decision Diagrams

Abstract

We describe an algorithm for solving MDPs with large state and action spaces, represented as factored MDPs with factored action spaces. Classical algorithms for solving MDPs are not effective since they require enumerating all the states and actions. As such, model minimization techniques have been proposed, and specifically, we extend the previous work on model minimization algorithm for MDPs with factored state and action spaces. Using algebraic decision diagrams, we compactly represent blocks of states and actions that can be regarded equivalent. We describe the model minimization algorithm that uses algebraic decision diagrams, and show that this new algorithm can handle MDPs with millions of states and actions.