Data-oriented scheduling for PROOF

被引:2
|
作者
Xu, Neng [1 ]
Guan, Wen [1 ]
Wu, Sau Lan [1 ]
Ganis, Gerardo
机构
[1] Univ Wisconsin, Madison, WI 53706 USA
关键词
D O I
10.1088/1742-6596/331/1/012009
中图分类号
O57 [原子核物理学、高能物理学];
学科分类号
070202 ;
摘要
The Parallel ROOT Facility - PROOF - is a distributed analysis system optimized for I/O intensive analysis tasks of HEP data. With LHC entering the analysis phase, PROOF has become a natural ingredient for computing farms at Tier3 level. These analysis facilities will typically be used by a few tenths of users, and can also be federated into a sort of analysis cloud corresponding to the Virtual Organization of the experiment. Proper scheduling is required to guarantee fair resource usage, to enforce priority policies and to optimize the throughput. In this paper we discuss an advanced priority system that we are developing for PROOF. The system has been designed to automatically adapt to unknown length of the tasks, to take into account the data location and availability (including distribution across geographically separated sites), and the {group, user} default priorities. In this system, every element - user, group, dataset, job slot and storage - gets its priority and those priorities are dynamically linked with each other. In order to tune the interplay between the various components, we have designed and started implementing a simulation application that can model various type and size of PROOF clusters. In this application a monitoring package records all the changes of them so that we can easily understand and tune the performance. We will discuss the status of our simulation and show examples of the results we are expecting from it.
引用
收藏
页数:7
相关论文
共 50 条
  • [1] Data-Oriented Runtime Scheduling Framework on Multi-GPUs
    Li, Tao
    Zhao, Kezhao
    Dong, Qiankun
    Leng, Jiabing
    Yang, Yulu
    Ma, Wenjing
    [J]. 2016 IEEE TRUSTCOM/BIGDATASE/ISPA, 2016, : 1311 - 1318
  • [2] Data-oriented parsing
    Klein, D
    [J]. COMPUTATIONAL LINGUISTICS, 2004, 30 (02) : 240 - 244
  • [3] A Theoretical Approach to the Data-Oriented Scheduling Strategies across Multiple Clouds
    Ma, Yongzheng
    Nan, Kai
    [J]. 2013 IEEE SIXTH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (CLOUD 2013), 2013, : 942 - 943
  • [4] Design of a Data-Oriented GPC
    Guan, Zhe
    Wakitani, Shin
    Yamamoto, Toru
    [J]. 2013 INTERNATIONAL CONFERENCE ON ADVANCED MECHATRONIC SYSTEMS (ICAMECHS), 2013, : 555 - 558
  • [5] On establishing and task scheduling of data-oriented vNF-SCs in an optical DCI
    Xu, Zichen
    Zhu, Zuqing
    [J]. JOURNAL OF OPTICAL COMMUNICATIONS AND NETWORKING, 2022, 14 (03) : 89 - 99
  • [6] Data-oriented language processing
    Bod, R
    Scha, R
    [J]. CORPUS-BASED METHODS IN LANGUAGE AND SPEECH PROCESSING, 1997, 2 : 137 - 173
  • [7] Data-Oriented Transaction Execution
    Pandis, Ippokratis
    Johnson, Ryan
    Hardavellas, Nikos
    Ailamaki, Anastasia
    [J]. PROCEEDINGS OF THE VLDB ENDOWMENT, 2010, 3 (01): : 928 - 939
  • [8] DATA-ORIENTED EXCEPTION HANDLING
    CUI, Q
    GANNON, J
    [J]. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 1992, 18 (05) : 393 - 401
  • [9] Design of a data-oriented evolutionary controller
    Fu, Qiuhao
    Koiwai, Kazushige
    Yamamoto, Toru
    [J]. IEEJ Transactions on Electronics, Information and Systems, 2017, 137 (07): : 914 - 915
  • [10] Efficient multivariate data-oriented microaggregation
    Josep Domingo-Ferrer
    Antoni Martínez-Ballesté
    Josep Maria Mateo-Sanz
    Francesc Sebé
    [J]. The VLDB Journal, 2006, 15 : 355 - 369