Approximating optimally discrete probability distribution with kth-order dependency for combining multiple decisions

Abstract

A probabilistic combination of K classifiers' decisions obtained from samples needs a (K + 1)st-order probability distribution. Chow and Liu (1968) as well as Lewis (1959) proposed an approximation scheme of such a high-order distribution with a product of only first-order tree dependencies. However, if a classifier follows more than two classifiers, such first-order dependency does not estimate adequately a high-order distribution. Therefore, a new method is proposed to approximate optimally the (K + 1)st-order distribution with a product set of kth-order dependencies where 1 less than or equal to k less than or equal to K, which are identified by a systematic dependency-directed approach. And also, a new method is presented to combine probabilistically multiple decisions with the product set of the kth-order dependencies, using a Bayesian formalism.