Estimating the Negative Binomial Dispersion Parameter with a Stratum-Effects Model and Many Strata

We investigate several estimation methods based on marginal and conditional likelihoods to estimate the negative binomial (NB) dispersion parameter for highly stratified count data, for which the statistical model has a separate mean parameter for each stratum. If the number of samples per stratum is small, then the model is highly parameterized, and the maximum likelihood estimator of the NB dispersion parameter can be seriously biased and inefficient. For marginal likelihoods, we assume either a lognormal or beta prior for functions of strata means. We demonstrate using simulations that the marginal and conditional likelihood-based estimators give much improved estimates compared to other methods for highly stratified count data, such as the double-extended quasi-likelihood estimator and the restricted maximum likelihood estimator. We prefer the conditional approach that does not rely on assumptions about the distribution of stratum means; however, this estimator may be less efficient in some situations. We demonstrate in a case study that these estimators can give substantially different results. We also provide simulation results about the power of likelihood ratio tests for change in the NB over-dispersion parameter. Supplementary materials accompanying this paper appear on-line.