Tidy Cache: Improving Data Placement in Die-stacked DRAM Caches

Die-stacked DRAM caches are likely to become available in mainstream chips in the near future. DRAM caches are typically used as a last level shared cache behind the traditional hierarchy of on-chip SRAM caches. However, its internal organization differs from traditional caches as it is based on DRAM technology that provides significantly diverse access latencies depending on the state of its internal structures. Accesses that hit in the row-buffer require only one DRAM command and are significantly faster than those that require closing the row-buffer to load a new row to read from. Prior work has focused on maximizing row-buffer locality while maintaining high cache hit ratios. However, past designs do not consider performance problems that may arise due to interleaved accesses from different applications that compete for the shared DRAM resources, nor the different access patterns and locality characteristics that each of these applications may have. In this paper, we first identify performance pathologies that are specific to DRAM caches which arise due to the interference caused by interleaved accesses from multiple cores. We then propose Tidy Cache, a novel DRAM cache design that is able to ameliorate these performance pathologies by dynamically adapting the replacement policy for demanded data. Our performance evaluation results show that our design outperforms the state-of-the-art by 9.2% for multi-programmed SPEC workloads and by 16.7% for a set of TPC-H queries, mainly due to significantly better cache miss ratios.