On-The-Fly Capacity Planning

When resolving performance problems, a simple histogram of hot call stacks does not cut it, especially given the highly fluid nature of modern deployments. Why bother tuning, when adding a few CPUs via the management console will quickly resolve the problem? The findings of these tools are also presented without any sense of context: e. g. string conversion may be expensive, but only matters if it contributes greatly to the response time of user logins. Historically, these concerns have been the purview of capacity planning. The power of planners lies in their ability to weigh demand versus capacity, and to do so in terms of the important units of work in the application (such as user logins). Unfortunately, they rely on measurements of rates and latencies, and both quantities are difficult to obtain. Even if possible, when all is said and done, these planners only relate to the code as a black-box: but, why bother adding CPUs, when easy code changes will fix the problem? We present a way to do planning on-the-fly: with a few call stack samples taken from an already-running system, we predict the benefit of a proposed tuning plan. We accomplish this by simulating the effect of a tuning action upon execution speed and the way it shifts resource demand. To identify existing problems, we show how to generate tuning actions automatically, guided by the desire to maximize speedup without needless expense, and that these generated plans may span resource and code changes. We show that it is possible to infer everything needed from these samples alone: levels of resource demand and the units of work in the application. We evaluate our planner on a suite of microbenchmarks and a suite of 15,000 data sets that come from real applications running in the wild.