Evaluation of graphical diagnostics for assessing goodness of fit of logistic regression models

The aim of the current work was to evaluate graphical diagnostics for assessment of the fit of logistic regression models. Assessment of goodness of fit of a model to the data set is essential to ensure the model provides an acceptable description of the binary variables seen. For logistic regression the most common diagnostic used for this purpose is binning the data and comparing the empirical probability of the occurrence of a dependent variable with the model predicted probability against the mean covariate value in the bin. Although intuitively appealing this method, which we term simple binning, may not have consistent properties for diagnosing model problems. In this report we describe and evaluate two different diagnostic procedures, random binning and simplified Bayes marginal model plots. These procedures were assessed via simulation under three different designs. Design 1: studies which were balanced on binary variables and a continuous covariate. Design 2: studies that were balanced on binary variables but unbalanced on the continuous covariate. Design 3: studies that were unbalanced on both the binary variables and the covariate. Each simulated study consisted of 500 individuals. Thirty studies were simulated. The covariate of interest was dose which could range from 0 to 20 units. The data were simulated with the dose being related to the outcome according to an Emax model on the logit scale. A logit Emax model (correct model) and a logit linear model (wrong model) were fitted to all data sets. The performance of the above diagnostics, in addition to simple binning, was compared. For all designs the proposed diagnostics performed at least as well and in many instances better than simple binning. In case of design 1 random binning and simple binning are identical. In the case of designs 2 and 3 random binning and simplified Bayes marginal model plots were superior in assessing the model fit when compared to simple binning. For the examples tested, both random binning and simplified Bayesian marginal model plots performed acceptably.