Monte-Carlo Tree Search in Continuous Action Spaces with Value Gradients

Cited 1 time in webofscience Cited 0 time in scopus
  • Hit : 55
  • Download : 0
Monte-Carlo Tree Search (MCTS) is the state-of-the-art online planning algorithm for large problems with discrete action spaces. However, many real-world problems involve continuous action spaces, where MCTS is not as effective as in discrete action spaces. This is mainly due to common practices such as coarse discretization of the entire action space and failure to exploit local smoothness. In this paper, we introduce Value-Gradient UCT (VG-UCT), which combines traditional MCTS with gradient-based optimization of action particles. VG-UCT simultaneously performs a global search via UCT with respect to the finitely sampled set of actions and performs a local improvement via action value gradients. In the experiments, we demonstrate that our approach outperforms existing MCTS methods and other strong baseline algorithms for continuous action spaces.
Association for the Advancement of Artificial Intelligence
Issue Date

34th AAAI Conference on Artificial Intelligence (AAAI 2020), pp.4561 - 4568

Appears in Collection
RIMS Conference Papers
Files in This Item
There are no files associated with this item.
This item is cited by other documents in WoS
⊙ Detail Information in WoSⓡ Click to see webofscience_button
⊙ Cited 1 items in WoS Click to see citing articles in records_button


  • mendeley


rss_1.0 rss_2.0 atom_1.0