Comments on the relationship between principal components analysis and weighted linear regression for bivariate data sets

Regression and principal components analysis (PCA) are two of the most widely used techniques in chemometrics. In this paper, these methods are compared by considering their application to linear, two-dimensional data sets with a zero intercept, The need for accommodating measurement errors with these methods is addressed and various techniques to accomplish this are considered. Seven methods are examined: ordinary least squares (OLS), weighted least squares (WLS), the effective variance method (EVM), multiply weighted regression (MWR), unweighted PCA (UPCA), and two forms of weighted PCA. Additionally, five error structures in x and y are considered: homoscedastic equal, homoscedastic unequal, proportional equal, proportional unequal, and random. It is shown that for certain error structures, several of the methods are mathematically equivalent. Furthermore, it is demonstrated that all of the methods can be unified under the principle of maximum likelihood estimation, embodied in the general case by MWR. Extensive simulations show that MWR produces the most reliable parameter estimates in terms of bias and mean-squared error. Finally, implications for modeling in higher dimensions are considered.