Efficient Online Performance Monitoring of Computing Systems using Predictive Models

Performance monitoring of datacenters provides vital information for dynamic resource provisioning, anomaly detection, capacity planning, and metering decisions. Online monitoring, however, incurs a variety of costs: the very act of monitoring a system interferes with its performance, consuming network bandwidth and disk space. With the goal of reducing these costs, we develop and validate a strategy based on exploiting the underlying structure of the signal being monitored to sparsify it prior to transmission to a monitoring station for analysis and logging. Specifically, predictive models are designed to estimate the signals of interest. These models are then used to obtain prediction errors-the error between the signal and the corresponding estimate-that are then treated as a sparse representation of the original signal while retaining key information. This transformation allows for far less data to be transmitted to the monitoring station, at which point the signal is reconstructed by simply using the prediction errors. We show that classical techniques such as principal component analysis (PCA) can be applied to the reconstructed signal for anomaly detection. Experimental results using the Trade6 and RuBBoS benchmarks indicate a significant reduction in overall transmission costs-greater that 95% in some cases-while retaining sufficient detection accuracy.