Unifying least squares, total least squares and data least squares

The standard approaches to solving overdetermined linear systems Ax approximate to b construct minimal corrections to the vector b and/or the matrix A such that the corrected system is compatible. In ordinary least squares (LS) the correction is restricted to b, while in data least squares (DLS) it is restricted to A. In scaled total least squares (Scaled TLS) [15], corrections to both b and A are allowed, and their relative sizes depend on a parameter gamma. Scaled TLS becomes total least squares (TLS) when gamma = 1, and in the limit corresponds to LS when gamma --> 0, and DLS when gamma --> infinity. In [13] we presented a particularly useful formulation of the Scaled TLS problem, as well as a new assumption that guarantees the existence and uniqueness of meaningful Scaled TLS solutions for all parameters gamma > 0, making the whole Scaled TLS theory consistent. This paper refers to results in [13] and is mainly historical, but it also gives some simpler derivations and some new theory. Here it is shown how any linear system Ax approximate to b can be reduced to a minimally dimensioned core system satisfying our assumption. The basics of practical algorithms for both the Scaled TLS and DLS problems are indicated for either dense or large sparse systems.