Evaluating the Impact of Inaccurate Information in Utility-Based Scheduling

Proponents of utility-based scheduling policies have shown the potential for a 100-1400% increase in value-delivered to users when used in lieu of traditional approaches such as FCFS, backfill or priority queues. However, perhaps due to concerns about their potential fragility, these policies are rarely used in practice. We present an evaluation of a utility-based scheduling policy based upon real workload data from both an auction-based resource infrastructure, and a supercomputing cluster. We model potential sources of imperfect operating conditions for a utility-based policy: user uncertainty and wealth inequity. Through simulation, we find that while the value delivered by a utility-based policy can degrade to half that of traditional approaches in the worst case, the policy we study provides 20-100% improvement under realistic operating conditions. We conclude that future efforts in designing utility-based allocation mechanisms and policies must explicitly consider the fidelity of elicited job value information from users.