Strategies for Fast I/O Throughput in Large-scale Climate Modeling Applications

Large-scale HPC applications are highly data-intensive with significant times spent in I/O operations. Many large-scale scientific applications do not adequately optimize the I/O operations, leading to overall poor performance. In this work, we have developed two main strategies for providing fast I/O throughput for an important climate modeling application, namely, Regional Ocean Modeling System (ROMS) that uses NetCDF for I/O operations. The strategies include load balancing the I/O operations and selective writing of data. We have also implemented file striping to improve I/O performance. Our experiments with up to 1440 processor cores and 5 days of simulations showed that our load balancing strategy resulted in about 27% decrease in execution times over the default executions, our selective writing strategy resulted in a further decrease of about 30% and the optimized file striping resulted in a further decrease of about 12% in execution times. All the strategies combined together improved the overall performance of the application by about 70%.