Estimate-based goodness-of-fit test for large sparse multinomial distributions

Abstract

The Pearson's chi-squared statistic (X(2)) does not in general follow a chi-square distribution when it is used for goodness-of-fit testing for a multinomial distribution based on sparse contingency table data. We explore properties of [Zelterman, D., 1987. Goodness-of-fit tests for large sparse multinomial distributions. J. Amer. Statist. Assoc. 82 (398), 624-629] D(2) statistic and compare them with those of X(2) and compare the power of goodness-of-fit test among the tests using D(2), X(2), and the statistic (L,) which is proposed by [Maydeu-Olivares, A., Joe, H., 2005. Limited- and full-information estimation and goodness-of-fit testing in 2(n) contingency tables: A unified framework. J. Amer. Statist. Assoc. 100 (471), 1009-1020] when the given contingency table is very sparse. We show that the variance of D(2) is not larger than the variance of X(2) under null hypotheses where all the cell probabilities are positive, that the distribution of D(2) becomes more skewed as the multinomial distribution becomes more asymmetric and sparse, and that, as for the L(r) statistic, the power of the goodness-of-fit testing depends on the models which are selected for the testing. A simulation experiment strongly recommends to use both D(2) and L, for goodness-of-fit testing with large sparse contingency table data. (C) 2008 Elsevier B.V. All rights reserved.