An Agent-Based Distributed Monitoring Framework

In compute clusters, monitoring of infrastructure and application components is essential for performance assessment, failure detection, problem forecasting, better resource allocation, and several other reasons. Present day trends towards larger and more heterogeneous clusters, rise of virtual data-centers, and greater variability of usage suggest that we have to rethink how we do monitoring. We need solutions that will remain scalable in the face of unforeseen expansions, can work in a wide-range of environments, and be adaptable to changes of requirements. We have developed an agent-based framework for constructing such monitoring solutions. Our framework deals with all scalability and flexibility issues associated with monitoring and leaves only the use-case specific task of data generation to the specific solution. This separation of concerns provides a versatile design that enables a single monitoring solution to work in a range of environments; and, at the same time, enables a range of monitoring solutions exhibiting different behaviors to be constructed by varying the tunable parameters of the framework. This paper presents the design, implementation, and evaluation of our novel framework.