A cooperative conjugate gradient method for linear systems permitting efficient multi-thread implementation

This paper revisits, in a multi-thread context, the so-called multi-parameter or block conjugate gradient (B-CG) methods, first proposed as sequential algorithms by O'Leary and Brezinski, for the solution of the linear system Ax = b, for an n-dimensional symmetric positive definite matrix A. Instead of the scalar parameters of the classical CG algorithm, which minimizes a scalar functional at each iteration, multiple descent and conjugate directions are updated simultaneously. Implementation involves the use of multiple threads and the algorithm is referred to as cooperative CG (CCG) to emphasize that each thread now uses information that comes from the other threads. It is shown that for a sufficiently large matrix dimension n, the use of an optimal number of threads results in a worst case flop count of O (n(7/3)) in exact arithmetic. Numerical experiments on a multi-core, multi-thread computer, for synthetic and real matrices, illustrate the theoretical results.