A robust, efficient, and balanced parallel algorithm for finding connected components

被引:0
|
作者
Asokan, M. [1 ]
机构
[1] Syncsort Inc, Pearl River, NY 10965 USA
关键词
Graph Mining; Connected Components; Hadoop; MapReduce; MAPREDUCE;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Finding connected components in an undirected graph has many practical applications. For example in a graph representing a social network, a connected component represents a group of related individuals with common interest. Also, finding connected components forms the basis for other clustering algorithms. In this paper, we will present a parallel algorithm which uses the well known sequential algorithm as the basis for finding connected components in an undirected graph. The algorithm can be adopted to run on a single computer with multiple cores or MapReduce. It is robust in the sense that it honors memory limits. This is important in today's containerized environments. It balances the workload even in the presence of data skew. For the best known algorithm running in MapReduce, the number of iterations is the square of the logarithmic function of the number of vertices in the graph. For our algorithm, we will prove that the upper bound on the number of iterations is a logarithmic function of the maximum size of a connected component. In each iteration, the amount of data read from or written to a file system is bounded by four times the number of edges in the graph.
引用
下载
收藏
页码:2113 / 2118
页数:6
相关论文
共 50 条
  • [41] PARALLEL ASYNCHRONOUS CONNECTED COMPONENTS IN A MESH
    HAMBRUSCH, S
    LUBY, M
    INFORMATION PROCESSING LETTERS, 1991, 38 (05) : 257 - 263
  • [42] COMPUTING CONNECTED COMPONENTS ON PARALLEL COMPUTERS
    HIRSCHBERG, DS
    CHANDRA, AK
    SARWATE, DV
    COMMUNICATIONS OF THE ACM, 1979, 22 (08) : 461 - 464
  • [43] On identifying strongly connected components in parallel
    Fleischer, LK
    Hendrickson, B
    Pinar, A
    PARALLEL AND DISTRIBUTED PROCESSING, PROCEEDINGS, 2000, 1800 : 505 - 511
  • [44] Finding strongly connected components in distributed graphs
    McLendon, W
    Hendrickson, B
    Plimpton, SJ
    Rauchwerger, L
    JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING, 2005, 65 (08) : 901 - 910
  • [45] ON FINDING THE STRONGLY CONNECTED COMPONENTS IN A DIRECTED GRAPH
    NUUTILA, E
    SOISALONSOININEN, E
    INFORMATION PROCESSING LETTERS, 1994, 49 (01) : 9 - 14
  • [46] Efficient GPU algorithms for parallel decomposition of graphs into strongly connected and maximal end components
    Wijs, Anton
    Katoen, Joost-Pieter
    Bosnacki, Dragan
    FORMAL METHODS IN SYSTEM DESIGN, 2016, 48 (03) : 274 - 300
  • [47] A Scalable Parallel Algorithm for Balanced Sampling
    Lee, Alexander
    Walzer-Goldfeld, Stefan
    Zablah, Shukry
    Riondato, Matteo
    THIRTY-SIXTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FOURTH CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE / TWELVETH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2022, : 12991 - 12992
  • [48] Efficient GPU algorithms for parallel decomposition of graphs into strongly connected and maximal end components
    Anton Wijs
    Joost-Pieter Katoen
    Dragan Bošnački
    Formal Methods in System Design, 2016, 48 : 274 - 300
  • [49] An Efficient Parallel Topic-Sensitive Expert Finding Algorithm Using Spark
    Yang, Yao-Ming
    Wang, Chang-Dong
    Lai, Jian-Huang
    2016 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2016, : 3556 - 3562
  • [50] Searching for backbones - An efficient parallel algorithm for finding groundstates in spin glass models
    Schneider, JJ
    SLOW DYNAMICS IN COMPLEX SYSTEMS, 2004, 708 : 426 - 429