Solving separable nonlinear least squares problems using the QR factorization

We present a method for solving the separable nonlinear least squares problem min(y,z)parallel to F(y, z)parallel to, where F(y, z) A(y)z b(y) with a full rank matrix A(y)is an element of R(N+l)xN, y is an element of R-n, z is an element of R-N and the vector b(y) is an element of RN+l , with small l >= n. We show how this problem can be reduced to a smaller equivalent problem min(y)parallel to f(y)parallel to where the function f has only components. The reduction technique is based on the existence of a locally differentiable orthonormal basis for the nullspace of A(T) (y). We use Newton's method to solve the reduced problem. We show that successive iteration points are independent of the nullspace basis used at any particular iteration point; thus the QR factorization can be used to provide a local basis at each iteration. We show that the first and second derivative terms that arise are easily computed, so quadratic convergence is obtainable even for nonzero residual problems. For the class of problems with N much greater than n and t the main cost per iteration of the method is one QR factorization of A(y). We provide a detailed algorithm and some numerical examples to illustrate the technique. Published by Elsevier B.V.