Updated Software Reliability Metrics

The Institute of Electrical and Electronics Engineers' Standard Dictionary of Measures of the Software Aspects of Dependability (982.1) has been scheduled for updating. This standard was issued several years ago. Since then new software reliability metrics have been developed and evaluated. In addition, modifications of metrics in the standard have developed and evaluated. The objective of this paper is to describe, evaluate, and apply the new and modified metrics, using failure data from several releases of the NASA Space Shuttle flight software. Recognizing that users of standards have other applications, the methodology, equations, and prediction plots are explained so that reliability engineers can apply the metrics to their applications. The metrics are assessed from two standpoints: 1) identify metrics that support a specified purpose (e.g., demonstrate reliability growth) and 2) use these metrics to identify software releases that, based on reliability predictions, are ready to deploy and identify which software requires additional testing. Prediction accuracy is computed for all metrics and the metrics are compared based on the results.