AdCoalescer: An Adaptive Coalescer to Reduce the Inter-Module Traffic in MCM-GPUs

The push for greater computing capabilities has led to the development of Multi-chip-module GPUs (MCM-GPUs), advancing parallel processing potential. Unfortunately, MCM-GPUs encounter a notable challenge, i.e., the performance bottleneck due to the inter-module network. Within MCM-GPUs, a large proportion of memory accesses by Streaming Multiprocessors (SMs) must traverse this inter-module network to access remote memory, encountering bandwidth constraints and increased latency-. This is in contrast to the efficient network-on-chip designs in single-module GPU architectures. In MCM-GPUs, we identify significant data access redundancy among SMs within a GPU module which can be coalesced to reduce the network pressure. However, directly coalescing by recording every memory address is inefficient, as a significant number of memory requests are directed to private data addresses, thus underutilizing the hardware resources. To address this challenge, we introduce the Adaptive Coalescer (AdCoalescer). AdCoalescer is a novel framework designed to adaptively coalesce memory requests from different SMs sent to the same cache lines, especially those likely to be concurrently accessed by multiple SMs. Our evaluations validate AdCoalescer design in alleviating the challenges posed by the inter-module network. On average, AdCoalescer achieves a performance improvement of 22.5% (with up to 71.9% improvement) compared to traditional designs with minimal hardware cost.