Robust performance evaluation of POMDP-based dialogue systems

Abstract

Partially observable Markov decision processes (POMDPs) have received significant interest in research on spoken dialogue systems, due to among many benefits its ability to naturally model the dialogue strategy selection problem under unreliable automated speech recognition. However, the POMDP approaches are essentially model-based, and as a result, the dialogue strategy computed from POMDP is still subject to the correctness of the user model. Moreover, there are difficulties in evaluating the dialogue strategies with no guarantee of the user model quality. In this paper, we propose a novel evaluation scheme for dialogue strategies of MDP/POMDP-based dialogue systems, which evaluates the dialogue strategies under model uncertainty. For the experiments, we extend some of the previous MDP user models to POMDPs, and evaluate the effects of user models on the dialogue strategy computed from POMDPs. We experimentally show that this evaluation method yields more reliable results in a sense that a strategy learned with a poor user model fails severely across different user models, and a strategy learned with a good user model performs well consistently. We additionally show that the strategy computed from POMDPs is more robust for model variations compared to MDPs.