Common framework for linear regression

This is a presentation of a common framework for linear regression that includes most linear regression methods based on linear algebra. The framework follows the guidelines of the H-principle. Modeling is carried out in steps that consist of two parts. The first part specifies the regression task. The second is the numerical algorithm, which is common for the different types of regression. This allows common treatment of different regression methods. There are two types of set-up: 1) where measurement data are used, and 2) where only variance/covariance matrices are used. The framework can be viewed as an extension of PLS regression to other types of regression analysis. Users can develop their own regression method that suits their views and preferences. Chemometric techniques, like cross-validation, test sets, dimension analysis and others, can be used in the same way for all regression methods within this framework. It is shown that by appropriate scaling of the computations, all regression methods within this framework provide numerically stable results. The second type of set-up uses a positive semi-definite matrix as an input. If a variance matrix is used as input, the algorithm can carry out PLS regression. If a regularized variance matrix from a ridge regression is used as input, ridge regression analysis can be carried out. Therefore, the same algorithm can be used for both PLS regression and ridge regression. Some properties of ridge regression and PLS regression are studied using the common algorithm. The common framework can be viewed as an extension of chemometric methodology to other types of linear regression. (C) 2015 Elsevier B.V. All rights reserved.