Container Orchestration on HPC Clusters

被引：8
|
作者：
Piras, Marco Enrico [1 ]
Pireddu, Luca [1 ]
Moro, Marco [1 ]
Zanetti, Gianluigi [1 ]
机构：
[1] CRS4, Loc Piscina Manna, I-09050 Loc Piscina Manna, Pula, Italy
来源：
HIGH PERFORMANCE COMPUTING: ISC HIGH PERFORMANCE 2019 INTERNATIONAL WORKSHOPS | 2020年 / 11887卷
关键词：
Cloud computing; HPC;
D O I：
10.1007/978-3-030-34356-9_3
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
Use of software containers and services in science is a rising trend that is not satisfied by the HPC computing resources often available in research contexts. We propose a method to grow Kubernetes clusters onto transient nodes allocated through the Grid Engine batch workload manager. The method is being used to run a mix of data-intensive service applications and bursty HPC-style workflows on an OpenStack-based Kubernetes deployment, while keeping a homogeneous job management, logging, monitoring, and storage infrastructure. Moreover, it is relatively straightforward to convert the implementation to be compatible with other workload managers.
引用
收藏
页码：25 / 35
页数：11
相关论文
共 50 条
  • [1] Container Orchestration on HPC Systems
    Zhou, Naweiluo
    Georgiou, Yiannis
    Zhong, Li
    Zhou, Huan
    Pospieszny, Marcin
    [J]. 2020 IEEE 13TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (CLOUD 2020), 2020, : 34 - 36
  • [2] Container orchestration on HPC systems through Kubernetes
    Zhou, Naweiluo
    Georgiou, Yiannis
    Pospieszny, Marcin
    Zhong, Li
    Zhou, Huan
    Niethammer, Christoph
    Pejak, Branislav
    Marko, Oskar
    Hoppe, Dennis
    [J]. JOURNAL OF CLOUD COMPUTING-ADVANCES SYSTEMS AND APPLICATIONS, 2021, 10 (01):
  • [3] Container orchestration on HPC systems through Kubernetes
    Naweiluo Zhou
    Yiannis Georgiou
    Marcin Pospieszny
    Li Zhong
    Huan Zhou
    Christoph Niethammer
    Branislav Pejak
    Oskar Marko
    Dennis Hoppe
    [J]. Journal of Cloud Computing, 10
  • [4] SGX-Aware Container Orchestration for Heterogeneous Clusters
    Vaucher, Sebastien
    Pires, Rafael
    Felber, Pascal
    Pasin, Marcelo
    Schiavoni, Valerio
    Fetzert, Christof
    [J]. 2018 IEEE 38TH INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS (ICDCS), 2018, : 730 - 741
  • [5] Enabling HPC workloads on Cloud Infrastructure using Kubernetes Container Orchestration Mechanisms
    Beltre, Angel
    Saha, Pankaj
    Govindaraju, Madhusudhan
    Younge, Andrew J.
    Grant, Ryan Eric
    [J]. PROCEEDINGS OF CANOPIE-HPC 2019:2019 IEEE/ACM 1ST INTERNATIONAL WORKSHOP ON CONTAINERS AND NEW ORCHESTRATION PARADIGMS FOR ISOLATED ENVIRONMENTS IN HPC (CANOPIE-HPC), 2019, : 11 - 20
  • [6] Evaluation of container orchestration systems for deploying and managing NoSQL database clusters
    Truyen, Eddy
    Van Landuyt, Dimitri
    Lagaisse, Bert
    Joosen, Wouter
    Bruzek, Matt
    [J]. PROCEEDINGS 2018 IEEE 11TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING (CLOUD), 2018, : 468 - 475
  • [7] Container Orchestration for Scientific Workflows
    Gerlach, Wolfgang
    Tang, Wei
    Wilke, Andreas
    Olson, Dan
    Meyer, Folker
    [J]. 2015 IEEE INTERNATIONAL CONFERENCE ON CLOUD ENGINEERING (IC2E 2015), 2015, : 377 - 378
  • [8] Ozy: A General Orchestration Container
    Osborne, Glenn
    Weninger, Tim
    [J]. 2016 IEEE INTERNATIONAL CONFERENCE ON WEB SERVICES (ICWS), 2016, : 609 - 616
  • [9] A container-based workflow for distributed training of deep learning algorithms in HPC clusters
    Gonzalez-Abad, Jose
    Lopez Garcia, Alvaro
    Kozlov, Valentin Y.
    [J]. CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2023, 26 (05): : 2815 - 2834
  • [10] High Performance MPI Library for Container-based HPC Cloud on InfiniBand Clusters
    Zhang, Jie
    Lu, Xiaoyi
    Panda, Dhabaleswar K.
    [J]. PROCEEDINGS 45TH INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING - ICPP 2016, 2016, : 268 - 277
12345