Performance-based parallel loop self-scheduling using hybrid OpenMP and MPI programming on multicore SMP clusters

Parallel loop self-scheduling on parallel and distributed systems has been a critical problem and it is becoming more difficult to deal with in the emerging heterogeneous cluster computing environments. In the past, some self-scheduling schemes have been proposed as applicable to heterogeneous cluster computing environments. In recent years, multicore computers have been widely included in cluster systems. However, previous researches into parallel loop self-scheduling did not consider certain aspects of multicore computers; for example, it is more appropriate for shared-memory multiprocessors to adopt Open Multi-Processing (OpenMP) for parallel programming. In this paper, we propose a performance-based approach using hybrid OpenMP and MPI parallel programming, which partition loop iterations according to the performance weighting of multicore nodes in a cluster. Because iterations assigned to one MPI process are processed in parallel by OpenMP threads run by the processor cores in the same computational node, the number of loop iterations allocated to one computational node at each scheduling step depends on the number of processor cores in that node. Experimental results show that the proposed approach performs better than previous schemes. Copyright (C) 2010 John Wiley & Sons, Ltd.