ERROR LOG ANALYSIS - STATISTICAL MODELING AND HEURISTIC TREND ANALYSIS

Most error log analysis studies perform a statistical fit to the data assuming a single underlying error process. This paper presents the results of an analysis that demonstrates the log is composed of at least two error processes: transient and intermittent. The mixing of data from multiple processes requires many more events to verify a hypothesis using traditional statistical analysis. Based on the shape of the interarrivai time function of the intermittent errors observed from actual error logs, a failure prediction heuristic, the Dispersion Frame Technique (DFT), is developed. The DFT was implemented in a distributed on-line monitoring and predictive diagnostic system for the campus-wide Andrew file system at Carnegie Mellon University. Data collected from 13 file servers over a 22 month period were analyzed using both the DFT and conventional statistical methods. It is shown that the DFT can extract intermittent errors from the error log and uses only one fifth of the error log entry points required by statistical methods for failure prediction. The DFT achieved a 93.7% success rate in failure prediction of both electromechanical and electronic devices. © 1990 IEEE