Making the case for reforming the I/O software stack of extreme-scale systems

The ever-increasing data needs of scientific and engineering applications require novel approaches to managing and exploring huge amounts of information in order to advance scientific discovery. In order to achieve this goal, one of the main priorities of the international scientific community is addressing the challenges of performing scientific computing on exascale machines within the next decade. Exascale platforms likely will be characterized by a three to four orders of magnitude increase in concurrency, a substantially larger storage capacity, and a deepening of the storage hierarchy. The current development model of independently applying optimizations at each layer of the system I/O software stack will not scale to the new levels of concurrency, storage hierarchy, and capacity. In this article we discuss the current development model for the I/O software stack of high-performance computing platforms. We identify the challenges of improving scalability, performance, energy efficiency, and resilience of the I/O software stack, while accessing a deepening hierarchy of volatile and nonvolatile storage. We advocate for radical new approaches to reforming the I/O software stack in order to advance toward exascale. (C) 2017 Elsevier Ltd. All rights reserved.