SYNCHRONIZATION AND COMMUNICATION COSTS OF LOOP PARTITIONING ON SHARED-MEMORY MULTIPROCESSOR SYSTEMS

To exploit loop level parallelism on shared memory multiprocessor systems, loops are decomposed and their execution scheduled on different processors in parallel. This paper presents strategies for static loop decomposition and scheduling as well as compiler assisted run-time scheduling that take into account, in addition to the cost of performing operations, the overhead costs associated with a decomposition and schedule. An algorithm for static decomposition of multidimensional loops based upon the operation execution costs, communication costs, and synchronization costs is discussed. Following the decomposition of a program, synchronization instructions are introduced to ensure correct program execution. An algorithm for determining the explicit synchronization instructions that should be introduced in a program to ensure correct execution of the program with arbitrarily nested loops is presented. Techniques for reducing run-time scheduling, communication and synchronization costs due to self scheduling, a compiler assisted run-time scheduling technique, of multidimensional loops are also presented. Experiments performed on the Encore multiprocessor system demonstrate that the techniques developed can reduce overhead costs.