Heuristic file sorted assignment algorithm of parallel I/O on cluster computing system

被引:0
|
作者
Zhi-gang Chen
Bi-qing Zeng
Ce Xiong
Xiao-heng Deng
Zhi-wen Zeng
An-feng Liu
机构
[1] Central South University,School of Information Science and Engineering
关键词
cluster computing; parallel I/O; file sorted assignment; variance of service time; TP338.8;
D O I
暂无
中图分类号
学科分类号
摘要
A new file assignment strategy of parallel I/O, which is named heuristic file sorted assignment algorithm was proposed on cluster computing system. Based on the load balancing, it assigns the files to the same disk according to the similar service time. Firstly, the files were sorted and stored at the set I in descending order in terms of their service time, then one disk of cluster node was selected randomly when the files were to be assigned, and at last the continuous files were taken orderly from the set I to the disk until the disk reached its load maximum. The experimental results show that the new strategy improves the performance by 20.2% when the load of the system is light and by 31.6% when the load is heavy. And the higher the data access rate, the more evident the improvement of the performance obtained by the heuristic file sorted assignment algorithm.
引用
收藏
页码:572 / 577
页数:5
相关论文
共 50 条
  • [41] Detection of I/O overloads in local HPC cluster Rocks with LUSTRE file system
    Edison Ramirez, John
    Rojas Cordero, Alexis
    [J]. 2017 CONGRESO INTERNACIONAL DE INNOVACION Y TENDENCIAS EN INGENIERIA (CONIITI), 2017,
  • [42] I/O response time in a fault-tolerant parallel virtual file system
    Feng, D
    Jiang, H
    Zhu, YF
    [J]. NETWORK AND PARALLEL COMPUTING, PROCEEDINGS, 2004, 3222 : 248 - 251
  • [43] Performance analysis of parallel file system for I/O load balancing in distributed applications
    Deepalakshmi, S
    Baswaraj, K
    Rao, SSSP
    Jagdish, BS
    Sonavane, DD
    Dhekne, PS
    Kaura, HK
    [J]. PDPTA'03: PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED PROCESSING TECHNIQUES AND APPLICATIONS, VOLS 1-4, 2003, : 713 - 720
  • [44] A heuristic algorithm for dynamic task scheduling in highly parallel computing systems
    Jovanovic, Z
    Maric, S
    [J]. FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE, 2001, 17 (06): : 721 - 732
  • [45] Parallel I/O Prefetching Using MPI File Caching and I/O Signatures
    Byna, Surendra
    Chen, Yong
    Sun, Xian-He
    Thakur, Rajeev
    Gropp, William
    [J]. INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS, 2008, : 350 - +
  • [46] A HEURISTIC TASK ASSIGNMENT ALGORITHM TO MAXIMIZE RELIABILITY OF A DISTRIBUTED SYSTEM
    HWANG, GJ
    TSENG, SS
    [J]. IEEE TRANSACTIONS ON RELIABILITY, 1993, 42 (03) : 408 - 415
  • [47] On evaluating decentralized parallel I/O scheduling strategies for parallel file systems
    Isaila, Florin
    Singh, David
    Carretero, Jesus
    Garcia, Felix
    [J]. HIGH PERFORMANCE COMPUTING FOR COMPUTATIONAL SCIENCE - VECPAR 2006, 2007, 4395 : 120 - +
  • [48] Enhanced file interoperability with parallel MPI File-I/O in image processing
    Piriyakumar, DAL
    Levi, P
    Rabenseifner, R
    [J]. RECENT ADVANCES IN PARALLEL VIRTUAL MACHINE AND MESSAGE PASSING INTERFACE, PROCEEDINGS, 2002, 2474 : 174 - 182
  • [49] pCFS vs. PVFS: Comparing a Highly-Available Symmetrical Parallel Cluster File System with an Asymmetrical Parallel File System
    Lopes, Paulo A.
    Medeiros, Pedro D.
    [J]. EURO-PAR 2010 PARALLEL PROCESSING, PT I, 2010, 6271 : 131 - +
  • [50] A New Approach to Double I/O performance for Ceph Distributed File System in Cloud Computing
    Zhang, Xiao
    Wang, Yanqiu
    Wang, Qing
    Zhao, Xiaonan
    [J]. 2019 2ND INTERNATIONAL CONFERENCE ON DATA INTELLIGENCE AND SECURITY (ICDIS 2019), 2019, : 68 - 75