Parallel solution of irregular, sparse matrix problems using High Performance Fortran

For regular, sparse, linear systems, like those derived from regular grids, using High Performance Fortran (HPF) for iterative solvers is straightforward. However, for irregular matrices the efficient implementation of solvers in HPF becomes much harder. First, the locality in the computations (a good partitioning) is unclear. Second, for efficiency we often use storage schemes that obscure even the simplest structure in the matrix (like rows and columns). Third, the limited capabilities of HPF to distribute data structures make it hard to implement the desired distribution. Fourth, data structures often have very different sizes and shapes, and matching the distributions for efficient implementation (locality) is a problem. Fifth, after implementing the distributions, we still must write the program in such a way that the compiler recognizes the efficient implementation and leaves out unnecessary communication, synchronization, etc. We discuss techniques for handling these problems, and our results demonstrate that efficient implementations are possible. In fact, we show that on larger numbers of processors the efficiency of our irregular, sparse matrix-vector product is higher than the efficiency of the inner product, another essential kernel in iterative methods. For comparison we show results for regular, sparse matrices. All our experiments are carried out using the Portland Group (PGI) HPF compiler (version 2.1) on the Intel Paragon at the Swiss Federal Institute of Technology (ETH Zurich).