Performance and Scalability Analysis for Parallel Reservoir Simulations on Three Supercomputer Architectures

In this work, we tested the performance and scalability on three supercomputers of different architectures including SDSC's Comet, SciNet's GPC and IBM's Blue Gene/Q systems, through benchmarking parallel reservoir simulations. The Comet and GPC systems adopt a fat-tree network and they are connected with InfiniBand interconnects technology. The Blue Gene/Q uses a 5-dimensional toroidal network and it is connected with custom interconnects. In terms of supercomputer architectures, these systems represent two main interconnect families: fat-tree and torus. To demonstrate the application scalability for supercomputers with today's diversified architectures, we benchmark a parallel black oil simulator that is extensively used in the petroleum industry. Our implementation for this simulator is coded in C and MPI, and it offers grids, data, linear solvers, preconditioners, distributed matrices and vectors and modeling modules. Load balancing is based on the Hilbert space-filling curve (HSFC) method. Krylov subspace and AMG solvers are implemented, including restarted GMRES, BiCGSTAB, and AMG solvers from Hypre. The results show that the Comet is the fastest supercomputer among tested systems and the Blue Gene/Q has the best parallel efficiency. The scalability analysis helps to identify the performance barriers for different supercomputer architectures. The study of testing the application performance serves to provide the insights for carrying out parallel reservoir simulations on large-scale computers of different architectures.