Enabling Efficient Inter-node Message Passing and Remote Memory Access via a uGNI-based Lightweight Network Substrate for Cray Interconnects

被引：0

作者：

Wickramasinghe, Udayanga ^{[1
]}

Lumsdaine, Andrew ^{[2
]}

机构：

[1] Indiana Univ, Sch Informat Comp & Engn, Bloomington, IN 47405 USA

[2] Pacific Northwest Natl Lab, Northwest Inst Adv Comp, Seattle, WA USA

来源：

2018 18TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID) | 2018年

关键词：

Inter-node communication; RMA; One-sided Message Passing;

D O I：

10.1109/CCGRID.2018.00006

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

Today's cutting edge network hardware features extremely low latency and high bandwidth transactions for higher-level communication substrates. The Cray XC/XE family of network fabrics, also known as Cray Aries/Gemini respectively, supports such high-performance remote memory access operations (RMA) and a plethora of transaction modes to optimize communication via lower-level interfaces such as uGNI and DMAPP. However, enabling efficient one-sided communication for higher-level substrates is difficult due to barriers presented by the programming model itself, as well as miscellaneous synchronization bottlenecks at the runtime layers. We present an efficient programming model based on a distributed memory allocator for RMA and a communication substrate based on readers and writers for inter-node message passing and RMA operations. We try to maximize performance by introducing a scalable RMA event notification scheme and synchronization protocols that fully leverage Aries/Gemini fabric. Micro-benchmark results demonstrate that our library outperforms Cray MPI-3.0-based RMA one-sided operations by 1.5X and up to 6X in certain cases and is comparable or improves upon performance on others.

引用

页码：578 / 588

页数：11