Robust Ridge Regression for High-Dimensional Data

Ridge regression, being based on the minimization of a quadratic loss function, is sensitive to outliers. Current proposals for robust ridge-regression estimators are sensitive to "bad leverage observations," cannot be employed when the number of predictors p is larger than the number of observations n, and have a low robustness when the ratio pin is large. In this article a ridge-regression estimate based on repeated M estimation ("MM estimation") is proposed. It is a penalized regression MM estimator, in which the quadratic loss is replaced by an average of rho(r(i)/(sigma) over cap), where r(i) are the residuals and (sigma) over cap the residual scale from an initial estimator, which is a penalized S estimator; and rho is a bounded function. The MM estimator can be computed for p > n and is robust for large p/n. A fast algorithm is proposed. The advantages of the proposed approach over its competitors are demonstrated through both simulated and real data. Supplemental materials are available online.