Parallel job scheduling with overhead: A benchmark study

We study parallel job scheduling, where each job may be scheduled on any number of available processors in a given parallel system. We propose a mathematical model to estimate a job's execution time when assigned to multiple parallel processors. The model incorporates both the linear computation speedup achieved by having multiple processors to execute a job and the overhead incurred due to communication, synchronization, and management of multiple processors working on the same job. We show that the model is sophisticated enough to reflect the reality in parallel job execution and meanwhile also concise enough to make theoretical analysis possible. In particular we study the validity of our overhead model by running well-known benchmarks on a parallel system with 1024 processors. We compare our fitting results with the traditional linear model without the overhead. The comparison shows conclusively that our model more accurately reflects the effect of the number of processors on the execution time. We also summarize some theoretical results for a parallel job schedule problem that uses our overhead model to calculate execution times.