HBench:Java']Java:: An application-specific benchmarking framework for Java']Java Virtual Machines

Java applications represent a broad class of programs, ranging from programs running on embedded products to high-performance server applications, Standard Java benchmarks ignore this fact and assume a fixed workload. When an actual application's behavior differs from that included in a standard benchmark, the benchmark results are useless, if not misleading. In this paper, we present HBench:Java, an application-specific benchmarking framework, based on the concept that a system's performance must be measured in the context of the application of interest, HBench:Java employs a methodology that uses vectors to characterize the application and the underlying Java Virtual Machine (JVM) and carefully combines the two vectors to form a single metric that reflects a specific application's performance on a particular JVM such that the performance of multiple JVMs can be realistically compared, Our performance results demonstrate HBench:Java's superiority over traditional benchmarking approaches in predicting relative performance of real applications and its ability to pinpoint performance problems, even with a simplified vector. Copyright (C) 2001 John Wiley & Sons, Ltd.