Communication policies performance: A case study

Communication activities are one of the most influential factors for the performance of parallel applications, and usually limit the number of processors that can be profitably allocated. Two components usually determine the communication cost of a parallel algorithm. One is the volume and range of data transfer which is inherent to a specific algorithm. The other is the choice of communication strategy, e.g., point-to-point versus collective exchanges, blocking versus non blocking protocols, which has impact on setup costs, overheads due to buffering and/or contentions Knowledge of comparative performance of different strategies can be very useful for a user if several choices are available. In this article we present the results of a study to determine the best approach to high volume, long range communications within the frame work of multidimensional FFT algorithm. We have investigated five widely used communication strategies, available in the MPI standard, which have identical data volumes and range of communications. We also present a systematic analysis of the causes of performance differences, with analytical models supporting the experimental evidence.