A scalability prediction approach for multi-threaded applications on manycore processors

In the manycore era, developing multi-threaded applications to efficiently leverage the increasing number of cores has become an emerging problem. However, each application can have different scalability because of the competition for shared resources, such as CPU cores, memory subsystem, or both, depending on the input set. Therefore, to obtain optimal performance of applications, it is crucial to dynamically predict the scalability of applications and allocate the appropriate number of threads to each application based on its scalability. In this paper, we propose bytes per instruction, which is a simple and effective model to provide insights into the scalability of multi-threaded applications, based on the analysis of the interactions among memory-level parallelism, instruction-level parallelism, and thread-level parallelism. Based on the BPI model, we propose (1) a classification approach and (2) scalability prediction algorithm for multi-threaded applications. Based on the scalability prediction algorithm, we implement the scalability-aware thread scheduling approach which can allocate the appropriate number of threads to optimize application performance. The evaluation results on a 61-core Intel Xeon Phi coprocessor show that our algorithm can predict the scalability of 120-, 180-, and 240-threaded applications with an average error of 6.8 %. Moreover, the accuracy of our prediction algorithm outperforms state-of-the-art instruction-level prediction and memory-level prediction by an average of 9.1  and 14.8 %, respectively. The scalability-aware thread scheduling approach outperforms full utilization by 12.7 %.