An Evaluation of the One-Sided Performance in Open MPI

Open MPI provides an implementation of the MPI-3.1 standard supporting native communication over a wide range of high-performance network interconnects. As of version 2.0.0 Open MPI provides two implementations of the MPI-3.1 Remote Memory Access (RMA) specification. One implementation uses point-to-point (Send/Recv) communication to emulate RMA operations. The other is a new optimized RMA implementation (osc/rdma) that makes use of native network Remote Direct Memory Access (RDMA) and Atomic Memory Operations (AMOs). The osc/rdma implementation required the extension and enhancement of the Byte Transport Layer (BTL) interface in Open MPI to provide true RMA support for both communication and synchronization. For this work, we present an overview and performance evaluation of the osc/rdma RMA implementation in Open MPI v2.0.0. We additionally present early performance numbers of additional optimizations for accumulate operations. We show that osc/rdma is comparable in performance to that of a tuned vendor supplied MPI, and in some cases can outperform the vendor MPI by as much as 2X.