Skew in Parallel Query Processing

被引:53
|
作者
Beame, Paul [1 ]
Koutris, Paraschos [1 ]
Suciu, Dan [1 ]
机构
[1] Univ Washington, Seattle, WA 98195 USA
关键词
Parallel Computation; Skew; Lower Bounds;
D O I
10.1145/2594538.2594558
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
We study the problem of computing a conjunctive query q in parallel, using p of servers, on a large database. We consider algorithms with one round of communication, and study the complexity of the communication. We are especially interested in the case where the data is skewed, which is a major challenge for scalable parallel query processing. We establish a tight connection between the fractional edge packing of the query and the amount of communication in two cases. First, in the case when the only statistics on the database are the cardinalities of the input relations, and the data is skew-free, we provide matching upper and lower bounds (up to a polylogarithmic factor of p) expressed in terms of fractional edge packings of the query q. Second, in the case when the relations are skewed and the heavy hitters and their frequencies are known, we provide upper and lower bounds expressed in terms of packings of residual queries obtained by specializing the query to a heavy hitter. All our lower bounds are expressed in the strongest form, as number of bits needed to be communicated between processors with unlimited computational power. Our results generalize prior results on uniform databases (where each relation is a matching) [4], and lower bounds for the MapReduce model [1].
引用
收藏
页码:212 / 223
页数:12
相关论文
共 50 条
  • [1] Parallel query processing in a polystore
    Pavlos Kranas
    Boyan Kolev
    Oleksandra Levchenko
    Esther Pacitti
    Patrick Valduriez
    Ricardo Jiménez-Peris
    Marta Patiño-Martinez
    [J]. Distributed and Parallel Databases, 2021, 39 : 939 - 977
  • [2] Adaptive parallel query processing
    Tok, WH
    Zhao, L
    Bressan, S
    [J]. PDPTA'2001: PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED PROCESSING TECHNIQUES AND APPLICATIONS, 2001, : 590 - 597
  • [3] Parallel query processing in a polystore
    Kranas, Pavlos
    Kolev, Boyan
    Levchenko, Oleksandra
    Pacitti, Esther
    Valduriez, Patrick
    Jimenez-Peris, Ricardo
    Patino-Martinez, Marta
    [J]. DISTRIBUTED AND PARALLEL DATABASES, 2021, 39 (04) : 939 - 977
  • [4] Parallel Approach in RDF Query Processing
    Vajgl, Marek
    Parenica, Jan
    [J]. PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON NUMERICAL ANALYSIS AND APPLIED MATHEMATICS 2016 (ICNAAM-2016), 2017, 1863
  • [5] Integrating query processing with parallel languages
    Myers, Brandon
    Oskin, Mark
    Howe, Bill
    [J]. 2015 13TH IEEE INTERNATIONAL CONFERENCE ON DATA ENGINEERING WORKSHOPS (ICDEW), 2015, : 240 - 244
  • [6] SPARQL Query Parallel Processing: A Survey
    Feng, Jiaying
    Meng, Chenhong
    Song, Jiaming
    Zhang, Xiaowang
    Feng, Zhiyong
    Zou, Lei
    [J]. 2017 IEEE 6TH INTERNATIONAL CONGRESS ON BIG DATA (BIGDATA CONGRESS 2017), 2017, : 444 - 451
  • [7] Communication Steps for Parallel Query Processing
    Tardos, Eva
    [J]. JOURNAL OF THE ACM, 2017, 64 (06)
  • [8] Parallel query processing for OLAP in grids
    Kotowski, Nelson
    Lima, Alexandre A. B.
    Pacitti, Esther
    Valduriez, Patrick
    Mattoso, Marta
    [J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE, 2008, 20 (17): : 2039 - 2048
  • [9] Algorithmic Aspects of Parallel Query Processing
    Koutris, Paris
    Salihoglu, Semih
    Suciu, Dan
    [J]. SIGMOD'18: PROCEEDINGS OF THE 2018 INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 2018, : 1659 - 1664
  • [10] Architectures for parallel query processing on networks of workstations
    Dandamudi, SP
    Jain, G
    [J]. INTERNATIONAL SOCIETY FOR COMPUTERS AND THEIR APPLICATIONS 10TH INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED COMPUTING SYSTEMS, 1997, : 444 - 451