A Practical Approach For Workload-Aware Data Movement in Disaggregated Memory Systems

Memory disaggregation is a solid alternative to traditional server systems that can overcome memory scalability issues in next-generation HPC data centers. In a rack-level disaggregated system, multiple compute nodes with small local memory rely on remote memory pools (memory nodes) to fulfill their memory demands. An in-network memory manager manages remote memory address space and allocates it to compute nodes which can access the memory at cache-line granularity using coherent interconnects such as CXL (or GenZ). However, the memory access cost is significantly increased due to the presence of the network. Even though a page migration system can exploit the locality of memory accesses, accessing a remote page starves the block-level requests. Further, page migrations introduce additional overheads which combined with starvation may even degrade the performance. All these issues require systematic evaluation of disaggregated memory systems to achieve improved designs. This paper presents a hardware mechanism for workload-aware data movement between compute and memory pools that significantly reduces the memory access cost. Firstly, our design enables centralized hot-page migration in a multi-tiered disaggregated memory that is aware of access patterns for individual compute nodes. Secondly, we analyze the complexities of accessing a remote memory page and propose a novel solution to eliminate starvation by serving all the remote memory requests at cache block granularity and by sharing bandwidth between page and block memory requests. Lastly, we add extra hardware support to get rid of additional overheads in a page migration system. We evaluate our designs over a variety of multi-threaded benchmarks using a cycle-level simulator which is specially designed to simulate a disaggregated memory system. Our design performs 10% to 100% better than traditional RDMA-based disaggregated systems that access remote memory at page granularity and 5% to 35% better than baseline disaggregated systems that use coherent interconnects for block-level access.