A Flow Perspective on Nonlinear Least-Squares Problems

Just as the damped Newton method for the numerical solution of nonlinear algebraic problems can be interpreted as a forward Euler timestepping on the Newton flow equations, the damped Gauss-Newton method for nonlinear least squares problems is equivalent to forward Euler timestepping on the corresponding Gauss-Newton flow equations. We highlight the advantages of the Gauss-Newton flow and the Gauss-Newton method from a statistical and a numerical perspective in comparison with the Newton method, steepest descent, and the Levenberg-Marquardt method, which are respectively equivalent to Newton flow forward Euler, gradient flow forward Euler, and gradient flow backward Euler. We finally show an unconditional descent property for a generalized Gauss-Newton flow, which is linked to Krylov-Gauss-Newton methods for large-scale nonlinear least squares problems. We provide numerical results for large-scale problems: An academic generalized Rosenbrock function and a real-world bundle adjustment problem from 3D reconstruction based on 2D images.