Micro-architecture evaluation using performance vectors

Benchmarking is a widely used approach to measure computer performance. Current use of benchmarks only provides running times to describe the performance of a tested system. Glancing through these execution times provides little or no information about system strengths and weaknesses. A novel benchmarking methodology is proposed to identify key performance parameters; the methodology is based on measuring performance vectors. A performance vector is a vector of ratings that represents delivered performance of primitive operations of a system. Measuring the performance vector of a system in a typical user workload can be a tough problem. We show how the performance vector falls out of an equation consisting of dynamic instruction counts and execution times of benchmarks. We present a non-linear approach for computing the performance vector. The efficacy of the methodology is ascertained by evaluating the micro-architecture of the Sun SuperSPARC super-scalar processor using SPEC benchmarks. Results show interesting tradeoffs in the SuperSPARC and speak favorably of our methodology.