MDLoader: A Hybrid Model-driven Data Loader for Distributed Deep Neural Networks Training

被引：0

作者：

Bae, Jonghyun ^{[1
]}

Choi, Jong Youl ^{[2
]}

Pasini, Massimiliano Lupo ^{[2
]}

Mehta, Kshitij ^{[2
]}

Ibrahim, Khaled Z. ^{[1
]}

机构：

[1] Lawrence Berkeley Natl Lab, Berkeley, CA 94720 USA

[2] Oak Ridge Natl Lab, Oak Ridge, TN USA

来源：

2024 IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM WORKSHOPS, IPDPSW 2024 | 2024年

关键词：

One-sided communication; Collective communication; Graph Neural Network; Performance estimator;

D O I：

10.1109/IPDPSW63119.2024.00203

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

In this work, we propose MDLoader, a hybrid in-memory data loader for distributed deep neural networks. MDLoader introduces a model-driven performance estimator to automatically switch between one-sided and collective communication at runtime.

引用

页码：1193 / 1195

页数：3

共 50 条

[41] Combining Data-Driven and Model-Driven Approaches for Optimal Distributed Control of Standalone Microgrid
Ahangar, Parvaiz Ahmad
Lone, Shameem Ahmad
Gupta, Neeraj
SUSTAINABILITY, 2023, 15 (16)
[42] Model-Driven Development of Distributed Ledger Applications
Fraternali, Piero
Gonzalez, Sergio Luis Herrera
Frigerio, Matteo
Righetti, Mattia
DATABASE SYSTEMS FOR ADVANCED APPLICATIONS. DASFAA 2022 INTERNATIONAL WORKSHOPS, 2022, 13248 : 104 - 119
[43] A Model-driven Workflow for Distributed Microservice Development
Rademacher, Florian
Sorgalla, Jonas
Sachweh, Sabine
Zuendorf, Albert
SAC '19: PROCEEDINGS OF THE 34TH ACM/SIGAPP SYMPOSIUM ON APPLIED COMPUTING, 2019, : 1260 - 1262
[44] Automatic model-driven recovery in distributed systems
Joshi, KR
Hiltunen, MA
Sanders, WH
Schlichting, RD
24TH IEEE SYMPOSIUM ON RELIABLE DISTRIBUTED SYSTEMS, PROCEEDINGS, 2005, : 25 - 36
[45] Practical Dynamic Security Region Model: A Hybrid Physical Model-Driven and Data-Driven Approach
Ren, Junzhi
Zeng, Yuan
Qin, Chao
IEEE TRANSACTIONS ON POWER SYSTEMS, 2025, 40 (01) : 728 - 739
[46] On-Demand and Model-Driven Case Building Based on Distributed Data Sources
van der Pas, Mark
Dijkman, Remco
Akcay, Alp
Adan, Ivo
Walker, John
CASE-BASED REASONING RESEARCH AND DEVELOPMENT, ICCBR 2023, 2023, 14141 : 69 - 84
[47] Formal Model-Driven Design of Distributed Algorithms
Kuhnrich, Morten
ELECTRONIC NOTES IN THEORETICAL COMPUTER SCIENCE, 2009, 251 : 49 - 64
[48] Virtualization for Testing in Model-driven Distributed System
Kim, Youngheum
Lee, Seungyong
Kim, Seungbeom
2012 IEEE 75TH VEHICULAR TECHNOLOGY CONFERENCE (VTC SPRING), 2012,
[49] Probabilistic Model-Driven Recovery in Distributed Systems
Joshi, Kaustubh R.
Hiltunen, Matti A.
Sanders, William H.
Schlichting, Richard D.
IEEE TRANSACTIONS ON DEPENDABLE AND SECURE COMPUTING, 2011, 8 (06) : 913 - 928
[50] Scalable Data Parallel Distributed Training for Graph Neural Networks
Koyama, Sohei
Tatebe, Osamu
2022 IEEE 36TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM WORKSHOPS (IPDPSW 2022), 2022, : 699 - 707

← 1 2 3 4 5 →