Fine-Grained Management of Thread Blocks for Irregular Applications

Co-locating threads with complementary resource usage is a key strategy for improving throughput in parallel machines such as GPUs. However, the proliferation of irregular algorithms which change execution behavior dynamically makes optimal thread placement impossible when done statically or by profiling kernels as a whole. In this work, we characterize the performance loss associated with current thread block scheduling policies in GPU architectures. We then demonstrate that an extension of these strategies incorporating dynamic performance metrics such as memory and functional unit utilization at the thread block level as well as preemptive thread block swapping can improve throughput. We show that performance on irregular algorithms can be improved by an average increase of 17.1% over static profiling methods and 12.9% over dynamic strategies with no changes to legacy software and minimal hardware extensions which increase SRAM storage area by less than 0.5%.