Optimization of large-scale graph traversal for supercomputers

In the big data era, with the significant development of graph data, the demand for computing resources is growing rapidly. Supercomputers are applied to process large-scale graph data, which puts forward higher requirements for the storage and computing capabilities of supercomputers. In order to efficiently process large-scale graph data and evaluate the graph processing capabilities of the Tianhe supercomputer, in this paper we propose a graph traversal optimization technique for improving the efficiency of the benchmark program of Graph500, an important benchmark for evaluating graph processing capabilities of supercomputer. The technique mainly adopts the vertex sorting and priority caching strategy, where the vertices in the graph are sorted by degree in a descending order and some key vertices are stored in the cache of the core group of the Tianhe system. Therefore, this technique cuts down on invalid memory access and reduces the communication overhead between processes for maximizing the usage of the bandwidth for the supercomputer system. In order to validate graph traversal based on vertex sorting and buffering, an optimized graph500 version named VS-graph500 is customized for the Tianhe supercomputer, experimental results demonstrate that the VS-graph500 has a significant acceleration and good scalability in the supercomputers testing system, and attains a stable testing performance at 2547.13EGTEPS when the graph testing scale is 37, which is superior to the 7th in Graph500 list in June 2020. © 2021, The Editorial Board of Journal of Xidian University. All right reserved.