Calibration and refinement for classification trees

Abstract

The calibration of forecasts for a sequence of events has an extensive literature. Since calibration does not ensure 'good' forecasts, the notion of refinement was introduced to provide a structure into which methods for comparing well-calibrated forecasters could be embedded. In this paper we apply these two concepts, calibration and refinement, to tree-structured statistical probability prediction systems by viewing predictions in terms of the expected value of a response variable given the values of a set of explanatory variables. When all of the variables are categorical, we show that, under suitable conditions, branching at the terminal node of a tree by adding another explanatory variable yields a tree with more refined predictions. (C) 1998 Elsevier Science B.V. All rights reserved.