On the chi-square approximation to the exact distribution of goodness-of-fit statistics in multinomial models with composite hypotheses

Multinomial models are increasingly being used in psychology, and this use always requires estimating model parameters and testing goodness of fit with a composite null hypothesis. Goodness of fit is customarily tested with recourse to the asymptotic approximation to the distribution of the statistics. An assessment of the quality of this approximation requires a comparison with the exact distribution, but how to compute this exact distribution when parameters are estimated from the data appears never to have been defined precisely. The main goal of this paper is to compare two different approaches to defining this exact distribution. One of the approaches uses the marginal distribution and is, therefore, independent of the data; the other approach uses the conditional distribution of the statistics given the estimated parameters and, therefore, is data-dependent. We carried out a thorough study involving various parameter estimation methods and goodness-of-fit statistics, all of them members of the general class of power-divergence measures. Included in the study were multinomial models with three to five cells and up to three parameters. Our results indicate that the asymptotic distribution is rarely a good approximation to the exact marginal distribution of the statistics, whereas it is a good approximation to the exact conditional distribution only when the vector of expected frequencies is interior to the sample space of the multinomial distribution.