BES plus plus : HPC Profile Open Source C Implementation

While existing resource management software systems each have distinct and advanced capabilities, the way in which a user submits a basic job is generally similar across resource mangement systems. Recognizing this, the HPC Profile Working Group in the Open Grid Forum (OGF) has recently created a set of standards to define a common Web-services-based interface to resource management systems, thereby significantly enhancing interoperability. We describe the design and implementation of BES++, our open source implementation of the OGF HPC Profile. BES++ supports LSF, PBS and SGE resource managers and provides a C interface to further extend this software. In addition to complying with the OGF HPC Basic Profile specification, we support emerging OGF HPC Profile Extensions such as File Staging and Advanced Filter. We support job forwarding from one BES++ server to another, thereby enhancing potential transparency to the client. In addition, BES++ currently offers proof-of-concept support for legacy client tools such as qsub by translating legacy scripts into invocations of our BES++ client. We evaluate the effectiveness of BES++ through microbenchmarks, assessment of correctness/interoperability, and ability to parse and translate legacy scripts. To our knowledge, this research is the first to comprehensively report on the challenges, issues, and evaluation of the implementation of the OGF HPC Profile specifications.