Adaptive Caching Policies for Chiplet Systems Based on Reinforcement Learning

Chiplet packaging becomes a popular solution to integrate more hardware components. However, shared memory access across chiplets suffers from high miss penalty due to long route latency and low bandwidth of inter-chiplet interconnects. We observe that the aggregated last-level cache (LLC) miss penalty takes approximately 35% of time on data access, and that the miss is dominated by coherence miss as a result of shared reads and writes from other LLCs. To address this problem, we propose a caching manager which speculatively enforces (or discards) LLC caching via online reinforcement learning. On every invalidated cacheline, the caching manager receives the cacheline access features, evaluates the current caching policy, and makes the next caching policy adaptively. Experimental evaluation justifies that the caching manager can reduce more than 10% coherence miss and offers a 3% speedup against a state-of-the-art cache coherence protocol.