Speculative Paging for Future NVM Storage

The quest for greater performance and efficiency has driven modern cloud applications towards "in-memory" implementations, such as memcached and Apache Spark. Looking forward, however, the costs of DRAM, due to its low area density and high energy consumption, may make this trend unsustainable. Traditionally, OS paging system mechanisms were intended to bridge the gap between expensive, under-provisioned DRAM and inexpensive, dense storage, however, in the past twenty years the latency of storage, relative to DRAM became too great to overcome without significant performance impact. Recent NVM storage devices, such as Intel Optane drives and aggressive, 3D flash SSDs, may dramatically change the picture for OS paging. These newdrives are expected to provide much lower latency compared to the existing flash-based SSDs or traditional HDDs. Unfortunately, even these future NVM drives are still much too slow to replace DRAM, since the access latency of fast NVM storage is expected on the order of tens of microseconds, and they often require block-level access. Unlike traditional HDDs, for which the baseline OS paging policies are designed, these new SSDs place no penalty for "random" access and their access latency promises to be significantly less than traditional SSDs, thus arguing for a rearchitecting of the OS paging system. In this paper, we propose SPAN (Speculative PAging for future NVM storage), a software-only, OS swap-based, page management and prefetching scheme designed for emerging NVM storage. Unlike the baseline OS swapping mechanism, which is highly optimized for traditional spinning disks, SPAN leverages the inherent parallelism of NVM devices to proactivley fetch a set of pages from NVM storage to the small and fast main DRAM. In doing so, SPAN yields a speedup of similar to 18% versus swapping into the NVM with the baseline OS (similar to 50% of the performance lost by the baseline OS versus placing the entire working set in DRAM memory). The proposed technique thus enables the utilization of such hybrid systems for memory-hungry applications, lowering the memory cost while keeping the performance comparable to the DRAM-only system.