NVMCache: Wear-Aware Load Balancing NVM-based Caching for Large-Scale Storage Systems

Load balancing is critical for storage clusters to scale out and improve performance under skewed workloads. Multi-layer caching mechanism is proposed to improve load balancing for large-scale storage systems. However, because dynamic random-access memory (DRAM)-based cache is limited by capacity, volatility and chip cost, non-volatile devices, such as flash-based SSD and 3D Xpoint, are gradually being deployed as caches in large-scale storage systems. The emerging non-volatile memories such as PCM and ReRAM are expected to replace DRAM as caches in the future. Experimental evaluations show that while multi-layer caching mechanism works well for DRAM-based caching, it leads to write imbalance under mixed read-write workloads for non-volatile memory (NVM)-based caching. Existing mechanisms for DRAM-based cache do not consider the limited endurance and read-write asymmetry issues that are associated with NVM-based caches. Thus, how to effectively utilize the physical characteristics of NVMs and extend their lifespans has become the main challenge of NVM-based caching systems. In this study, we present NVMCache, an NVM-based multilayer caching mechanism. NVMCache introduces a wear-balancing scheme to avoid local intensive write operations to balance both load and wear. NVMCache aims to 1) ensure I/O load balancing and avoid access bottlenecks for large-scale storage systems, 2) take advantage of NVM's read/write asymmetry and prioritize access to reduce write-request blocking and improve overall access performance, and 3) extend the overall service lifetime of the NVM-based cache. The experimental results show that under different workloads, compared with DistCache, the state-of-the-art multi-layer caching mechanism, NVMCache improves throughput by a maximum of 10% and reduces wear imbalance by a maximum of 80%.