Shrinkage estimation and selection for a logistic regression model

This paper considers the problem of variable selection and the estimation for a logistic regression model via shrinkage and three penalty methods. We develop a large sample theory for the shrinkage estimators including asymptotic distributional bias and risk. We show that if the shrinkage dimension exceeds two, the asymptotic risk of the shrinkage estimator is strictly less than the classical estimators for a wide class of models. This reduction holds globally in the parameter space. Furthermore, we consider three different penalty estimators: the LASSO, adaptive LASSO, and SCAD and compare their relative performance with the shrinkage estimators numerically. A Monte Carlo simulation study is conducted for different combinations of inactive predictors and the performance of each method is evaluated in terms of a simulated mean squared error. This study indicates that shrinkage method is comparable to the LASSO, adaptive LASSO, and SCAD when the number of inactive predictors in the model is relatively large. A real data example is presented to illustrate the proposed methodologies.