A new model for the analysis of asynchronous parallel algorithms

The asynchronous nature of some MPI/PVM programs does not easily fit inside the BSP model. Through the suppression of barriers and the generalization of the concept of superstep we propose a new model, the BSP Without Barriers (BSPWB) model. The BSPWB parameters and their quality are evaluated on four standard parallel platforms: the GRAY T3E, the IBM SP2, the Origin 2000 and the Digital Alpha Server 8400. This study proves that the time spent in an h-relation is more independent on the number of processors than on the communication pattern. We illustrate the BSPWB model using two problem-solving paradigms: the Nested Parallel Recursive Divide and Conquer Paradigm and the Virtual Pipeline Dynamic Programming Paradigm. The proposed paradigms explain how nested parallelism and processor virtualization may be introduced in MPI and PVM without having any negative impact in the performance and model accuracy. The prediction of the communication times is robust even for problems where communication is dominated by small messages.