Dуnаmiс Inсrеmеntаl Grарh Pаrtitiоning Аlgоrithm Ваsеd оn vеrtеx Grоuр Rеdistributiоn

被引：0
|
作者：
Li H. [1 ]
Liu Y.-N. [1 ]
Yang S.-Q. [1 ]
Huang J.-B. [1 ]
Qiao S.-J. [2 ]
机构：
[1] School of Computer Science and Technology, Xidian University, Xi’an
[2] School of Software Engineering, Chengdu University of Information Technology, Chengdu
来源：
Ruan Jian Xue Bao/Journal of Software | 2024年 / 35卷 / 04期
关键词：
dynamic incremental graph partitioning algorithm; graph partitioning; local optimization;
D O I：
10.13328/j.cnki.jos.006842
中图分类号：
学科分类号：
摘要：
Graph partitioning is a basic task for distributed graph computing. It is used to divide a large-scale graph into different parts and allocate them to different machines in a cluster. The quality of graph partitioning has a great impact on the performance of distributed graph computing, and graph partitioning aims to minimize edge cuts and load balance. Nowadays, the graph data usually grow dynamically, which needs a partitioning method to process dynamic incremental graphs, so as to ensure the quality of graph partitioning. Although some dynamic graph partitioning algorithms have been presented recently, they cannot process real-time dynamic changes and obtain high-quality graph partitioning results simultaneously. In this study, a dynamic incremental graph partitioning algorithm based on vertex group redistribution (ED-IDGP) is proposed to solve the problem of large-scale dynamic incremental graph partitioning. In ED-IDGP, a dynamic processor is designed to process four different unit update types in real time, and the graph partitioning quality is further improved by executing a local optimizer near the dynamic change in the partition after each unit update. In the local optimizer of ED-IDGP, a vertex group search strategy based on the improved label propagation algorithm is used to search for the vertex group, and a vertex group movement gain formula is proposed to measure the most beneficial vertex group and move it to the target partition for optimization. This study evaluates the performance and efficiency of the ED-IDGP algorithm from different perspectives and metrics on real datasets. © 2024 Chinese Academy of Sciences. All rights reserved.
引用
收藏
页码：1819 / 1840
页数：21
相关论文
共 41 条
  • [1] Malewicz G, Austern MH, Bik AJC, Dehnert JC, Horn I, Leiser N, Czajkowski G., Pregel: A system for large-scale graph processing, Proc. of the 2010 ACM SIGMOD Int’l Conf. on Management of Data, pp. 135-146, (2010)
  • [2] (2021)
  • [3] Low Y, Bickson D, Gonzalez J, Guestrin C, Kyrola A, Hellerstein JM., Distributed GraphLab: A framework for machine learning and data mining in the cloud, Proc. of the VLDB Endowment, 5, 8, pp. 716-727, (2012)
  • [4] Chen R, Shi JX, Chen YZ, Zang BY, Guan HB, Chen HB., PowerLyra: Differentiated graph computation and partitioning on skewed graphs, ACM Trans. on Parallel Computing, 5, 3, (2018)
  • [5] Cui PJ, Yuan Y, Li CH, Zhang C, Wang GR., RGraph: Effective distributed graph data processing system based on RDMA, Ruan Jian Xue Bao/Journal of Software, 33, 3, pp. 1018-1042, (2022)
  • [6] Neo4j, (2022)
  • [7] Sarwat M, Elnikety S, He YX, Kliot G., Horton: Online query execution engine for large distributed graphs, Proc. of the 28th IEEE Int’l Conf. on Data Engineering, pp. 1289-1292, (2012)
  • [8] Kang U, Tong HH, Sun JM, Lin CY, Faloutsos C., Gbase: A scalable and general graph management system, Proc. of the 17th ACM SIGKDD Int’l Conf. on Knowledge Discovery and Data Mining, pp. 1091-1099, (2011)
  • [9] Andreev K, Racke H., Balanced graph partitioning, Theory of Computing Systems, 39, 6, pp. 929-939, (2006)
  • [10] Hendrickson B, Leland R., An improved spectral graph partitioning algorithm for mapping parallel computations, SIAM Journal on Scientific Computing, 16, 2, pp. 452-469, (1995)
12345