A Distributed Persistent Memory File System Based on RDMA Multicast

被引：0
|
作者：
Chen M. [1 ]
Zheng S. [2 ]
You L. [1 ]
Wang J. [1 ]
Yan T. [1 ]
Tu Y. [3 ]
Han Y. [3 ]
Huang L. [1 ]
机构：
[1] Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai
[2] Department of Computer Science and Technology, Tsinghua University, Beijing
[3] ZTE Corporation, Nanjing
来源：
Zheng, Sheng'an (venero@tsinghua.edu.cn) | 1600年 / Science Press卷 / 58期
关键词：
Distributed file system; Multicast; Persistent memory; Remote direct memory access; Remote procedure call;
D O I：
10.7544/issn1000-1239.2021.20200369
中图分类号：
学科分类号：
摘要：
The development of persistent memory and remote direct memory access(RDMA) provides new opportunities for designing efficient distributed systems. However, the existing RDMA-based distributed systems are far from fully exploiting RDMA multicast capabilities, which makes them difficult to solve the problem of multi-copy file data transmission in one-to-many transmission, degrading system performance. In this paper, a distributed persistent memory and RDMA multicast transmission based file system(MTFS) is proposed. It efficiently transmits data to different data nodes by the low-latency multicast transmission mechanism, which makes full use of the RDMA multicast capability, hence avoiding high latency due to multi-copy file data transmission operations. To improve the flexibility of transmission operations, a multi-mode multicast remote procedure call(RPC) mechanism is proposed, which enables the adaptive recognition of RPC requests, and moves transmission operations out of the critical path to further improve transmission efficiency. MTFS also provides a lightweight consistency guarantee mechanism. By designing a crash recovery mechanism, a data verification module and a retransmission scheme, MTFS is able to quickly recover from a crash, and achieves file system reliability and data consistency by error detection and data correction. Experimental results show that MTFS has greatly increased the throughput by 10.2-219 times compared with GlusterFS. MTFS outperforms NOVA by 10.7% on the Redis workload, and achieves good scalability in multi-thread workloads. © 2021, Science Press. All right reserved.
引用
收藏
页码：384 / 396
页数：12
相关论文
共 31 条
  • [1] Zhang Hongbin, Fan Jie, Shu Jiwu, Et al., Summary of storage system and technology based on phase change memory, Journal of Computer Research and Development, 51, 8, pp. 1647-1662, (2014)
  • [2] Shu Jiwu, Lu Youyou, Zhang Jiacheng, Et al., Research progress on non-volatile memory based storage system, Science & Technology Review, 34, 14, pp. 86-94, (2016)
  • [3] Qureshi M K, Srinivasan V, Rivers J A., Scalable high performance main memory system using phase-change memory technology, Proc of the 36th Annual Int Symp on Computer Architecture, pp. 24-33, (2009)
  • [4] Kultursay E, Kandemir M, Sivasubramaniam A, Et al., Evaluating STT-RAM as an energy-efficient main memory alternative, Proc of 2013 IEEE Int Symp on Performance Analysis of Systems and Software (ISPASS), pp. 256-267, (2013)
  • [5] Optane DC persistent memory, (2019)
  • [6] Xu Jian, Swanson S., NOVA: A log-structured file system for hybrid volatile/non-volatile main memories, Proc of the 14th USENIX Conf on File and Storage Technologies, pp. 323-338, (2016)
  • [7] Dong Mingkai, Chen Haibo, Soft updates made simple and fast on non-volatile memory, Proc of 2017 USENIX Annual Technical Conf, pp. 719-731, (2017)
  • [8] Dulloor S R, Kumar S, Keshavamurthy A, Et al., System software for persistent memory, Proc of the 9th European Conf on Computer Systems, pp. 1-15, (2014)
  • [9] Zheng Shengan, Huang Linpeng, Liu Hao, Et al., Hmvfs: A hybrid memory versioning file system, Proc of the 32nd Symp on Mass Storage Systems and Technologies, pp. 1-14, (2016)
  • [10] Wilcox M., Add support for NV-DIMMs to ext4, (2014)
1234