ITERATIVE WEIGHTED LEAST-SQUARES ESTIMATION IN HETEROSCEDASTIC LINEAR-MODELS

This article addresses the problem of choosing weights for iterative weighted least squares estimation in heteroscedastic linear models. An asymptotically optimal method for determining weights at each iteration is derived under a Bayesian model for the variances. The method uses a compromise between model-based and model-free variance estimates. Consider a heteroscedastic linear regression model in which responses are grouped so that the variance is constant within each group. Let beta denote the vector of regression parameters and let theta denote a vector of parameters determining a prior distribution for the variances. Iterative weighted least squares estimators are defined as follows. Given estimates beta and theta, calculate a weight for the ith group as a function of theta, the values in the ith group of the predictor variables, and the average of the squared residuals from the estimated mean responses in the ith group. Given weights, calculate the weighted least squares estimate beta and a new estimate theta. Continue until beta converges. We derive the asymptotically optimal weight function under an inverse gamma model for the variances. The resulting weights have a simple form. At each iteration the inverse weight for the ith group is a weighted average of the average squared residual and a variance estimate based on the inverse gamma model.