Adaptive job routing and scheduling

Computer systems are rapidly becoming so complex that maintaining them with human support staffs will be prohibitively expensive and inefficient. In response, visionaries have begun proposing that computer systems be imbued with the ability to configure themselves, diagnose failures, and ultimately repair themselves in response to these failures. However, despite convincing arguments that such a shift would be desirable, as of yet there has been little concrete progress made towards this goal. These challenges are naturally suited to machine learning methods. Hence, this article presents a new network simulator designed to study the application of machine learning methods from a system-wide perspective. Also, learning-based methods for addressing the problems of job routing and CPU scheduling in the simulated networks are introduced. Experimental results verify that methods using machine learning outperform reasonable heuristic and hand-coded approaches on example networks designed to capture many of the complexities that exist in real systems. (C) 2004 Elsevier Ltd. All rights reserved.