The practice of I/O optimizations for out-of-core computation

被引:0
|
作者
Tang, JQ [1 ]
Fang, BX [1 ]
Hu, MZ [1 ]
Zhang, HL [1 ]
机构
[1] Harbin Inst Technol, Dept Comp Sci & Engn, Harbin 150001, Peoples R China
关键词
D O I
暂无
中图分类号
TP301 [理论、方法];
学科分类号
081202 ;
摘要
Out-of-core computation needs high I/O performance as the out-of-core data stored in files has to be moved between memory and disks during the execution of an application program. In order to obtain better performance of out-of-core programs, two I/O optimizations are practiced, which are data prefetching and data reuse. This paper focuses on discussing the mechanism and implementation of each method and under which condition a method should be used. Several sets of experiments are taken to test the strategies and the results show that the I/O performance is efficiently improved by using thew optimization methods.
引用
收藏
页码:81 / 85
页数:5
相关论文
共 50 条
  • [41] Out-of-core divisible load processing
    Drozdowski, M
    Wolniewicz, P
    IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2003, 14 (10) : 1048 - 1056
  • [42] Out-of-core clustering of volumetric datasets
    GRANBERG Carl J.
    Journal of Zhejiang University-Science A(Applied Physics & Engineering), 2006, (07) : 1134 - 1140
  • [43] I/O Chunking and Latency Hiding Approach for Out-of-core Sorting Acceleration using GPU and Flash NVM
    Sato, Hitoshi
    Mizote, Ryo
    Matsuoka, Satoshi
    Ogawa, Hirotaka
    2016 IEEE INTERNATIONAL CONFERENCE ON BIG DATA (BIG DATA), 2016, : 398 - 403
  • [44] HUS-Graph: I/O-Efficient Out-of-Core Graph Processing with Hybrid Update Strategy
    Xu, Xianghao
    Wang, Fang
    Jiang, Hong
    Cheng, Yongli
    Feng, Dan
    Zhang, Yongxuan
    PROCEEDINGS OF THE 47TH INTERNATIONAL CONFERENCE ON PARALLEL PROCESSING, 2018,
  • [45] A Highly Efficient I/O-based Out-of-Core Stencil Algorithm with Globally Optimized Temporal Blocking
    Midorikawa, Hiroko
    Tan, Hideyuki
    2017 IEEE PACIFIC RIM CONFERENCE ON COMMUNICATIONS, COMPUTERS AND SIGNAL PROCESSING (PACRIM), 2017,
  • [46] SOWalker: An I/O-Optimized Out-of-Core Graph Processing System for Second-Order RandomWalks
    Wu, Yutong
    Shi, Zhan
    Huang, Shicai
    Tian, Zhipeng
    Zuo, Pengwei
    Fang, Peng
    Wang, Fang
    Feng, Dan
    PROCEEDINGS OF THE 2023 USENIX ANNUAL TECHNICAL CONFERENCE, 2023, : 87 - 100
  • [47] A linux cluster-based parallel I/O system for high performance and out-of-core volume rendering
    Jeong, KJ
    Kim, JI
    Kim, NK
    Kim, JH
    Ryu, YJ
    PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON PARALLEL AND DISTRIBUTED PROCESSING TECHNIQUES AND APPLICATIONS, VOLS I-V, 2000, : 2319 - 2324
  • [48] A compression-based memory-efficient optimization for out-of-core GPU stencil computation
    Shen, Jingcheng
    Long, Linbo
    Deng, Xin
    Okita, Masao
    Ino, Fumihiko
    JOURNAL OF SUPERCOMPUTING, 2023, 79 (10): : 11055 - 11077
  • [49] A compression-based memory-efficient optimization for out-of-core GPU stencil computation
    Jingcheng Shen
    Linbo Long
    Xin Deng
    Masao Okita
    Fumihiko Ino
    The Journal of Supercomputing, 2023, 79 : 11055 - 11077
  • [50] File I/O Cache Performance of Supercomputer Fugaku Using an Out-of-Core Direct Numerical Simulation Code of Turbulence
    Hatanaka, Yuto
    Yamane, Yuki
    Yamaguchi, Kenta
    Soga, Takashi
    Musa, Akihiro
    Ishihara, Takashi
    Uno, Atsuya
    Komatsu, Kazuhiko
    Kobayashi, Hiroaki
    Yokokawa, Mitsuo
    COMPUTATIONAL SCIENCE, ICCS 2024, PT VI, 2024, 14937 : 173 - 187