Application-level Determinism in Distributed Systems

被引:0
|
作者
Boelmann, Christopher [1 ]
Schwittmann, Lorenz [1 ]
Waltereit, Marian [1 ]
Wander, Matthaeus [1 ]
Weis, Torben [1 ]
机构
[1] Univ Duisburg Essen, Distributed Syst Grp, Duisburg, Germany
关键词
Application-level Determinism; Deterministic Distributed Systems; Reproducible Program Execution; Parallel Program Execution; Operational Transformation;
D O I
10.1109/ICPADS.2016.130
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Deterministic and reproducible program execution eases the development and debugging of distributed systems. However, deterministic execution comes at high performance costs and is hard to achieve, especially when running on different hardware. In this paper we introduce the concept of application-level determinism and describe how the parallel programming model Spawn & Merge can be used for scalable and deterministic distributed computation. Application-level deterministic applications yield reproducible deterministic results independent of the number of nodes participating in the computation, even though intermediate tasks may be executed in an unpredictable schedule. To achieve consistency independent of the order in which operations have been applied we present a new Operational Transformation algorithm, which mitigates the performance loss of introducing determinism with Spawn & Merge. We show that such deterministic processing can scale across a cluster of compute nodes and discuss for which kind of workload the programming model is feasible. Furthermore, for high and low workloads, we evaluate the cost of adding determinism to be 28% and 40% higher than perfect parallel computation.
引用
收藏
页码:989 / 998
页数:10
相关论文
共 50 条
  • [1] A Distributed Application-level IT System Workload Generator
    Mueller, Christopher
    Horie, Michael
    Neville, Stephen W.
    [J]. 2009 INTERNATIONAL CONFERENCE ON ADVANCED INFORMATION NETWORKING AND APPLICATIONS WORKSHOPS: WAINA, VOLS 1 AND 2, 2009, : 1100 - 1105
  • [2] Scaling Application-Level Dynamic Taint Analysis to Enterprise-Scale Distributed Systems
    Fu, Xiaoqin
    Cai, Haipeng
    [J]. 2020 ACM/IEEE 42ND INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: COMPANION PROCEEDINGS (ICSE-COMPANION 2020), 2020, : 270 - 271
  • [3] Application-level robustness and redundancy in linear systems
    Alippi, C
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-FUNDAMENTAL THEORY AND APPLICATIONS, 2002, 49 (07): : 1024 - 1027
  • [4] Specifying application-level security in workflow systems
    Olivier, MS
    van de Riet, RP
    Gudes, E
    [J]. NINTH INTERNATIONAL WORKSHOP ON DATABASE AND EXPERT SYSTEMS APPLICATIONS, PROCEEDINGS, 1998, : 346 - 351
  • [5] Infrastructural Support for Enforcing and Managing Distributed Application-Level Policies
    Goovaerts, Tom
    De Win, Bart
    Joosen, Wouter
    [J]. ELECTRONIC NOTES IN THEORETICAL COMPUTER SCIENCE, 2008, 197 (01) : 31 - 43
  • [6] Application-level web component framework for distributed workflow management
    Zhu, Yinzhou
    Yin, Baolin
    [J]. 2013, Science and Engineering Research Support Society, 20 Virginia Court, Sandy Bay, Tasmania, Australia (06) : 199 - 208
  • [7] Search-Based Parameter Tuning on Application-Level Load Balancing For Distributed Embedded Systems
    Caliskanelli, Ipek
    Indrusiak, Leandro Soares
    [J]. 2013 IEEE 15TH INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE COMPUTING AND COMMUNICATIONS & 2013 IEEE INTERNATIONAL CONFERENCE ON EMBEDDED AND UBIQUITOUS COMPUTING (HPCC_EUC), 2013, : 2050 - 2057
  • [8] Application-level QoS
    Woodside, Murray
    Menasce, Daniel A.
    [J]. IEEE INTERNET COMPUTING, 2006, 10 (03) : 13 - 15
  • [9] Cachematic - Automatic Invalidation in Application-Level Caching Systems
    Holmqvist, Viktor
    Nilsfors, Jonathan
    Leitner, Philipp
    [J]. PROCEEDINGS OF THE 2019 ACM/SPEC INTERNATIONAL CONFERENCE ON PERFORMANCE ENGINEERING (ICPE '19), 2019, : 167 - 178
  • [10] Application-level fairness
    Kangasharju, Jussi
    Mu, Mu
    Colussi, Gian Donato
    [J]. 2008 THE INTERNATIONAL CONFERENCE ON INFORMATION NETWORKING, 2008, : 340 - +