The necessity of assuring quality in software measurement data

Software measurement data is often used to model software quality classification models. Related literature has focussed on developing new classification techniques and schemes with the aim of improving classification accuracy. However, the quality of software measurement data used to build such classification models plays a critical role in their accuracy and usefulness. We present empirical case studies which demonstrate that despite using a very large number of diverse classification techniques for building software quality classification models, the classification accuracy does not show a dramatic improvement. For example, a simple lines-of-code based classification performs comparatively to some other more advanced classification techniques such as neural networks, decision trees, and case-based reasoning. Case studies of the NASA JM1 and KC2 software measurement datasets (obtained through the NASA Metrics Data Program) are presented. Some possible reasons that affect the quality of a software measurement dataset include presence of data noise, errors due to improper software data collection, exclusion of software metrics that are better representative software quality indicators, and improper recording of software fault data. This study shows, through an empirical study, that instead of searching for a classification technique that will perform well for a given software measurement dataset, the software quality and development teams should focus on improving the quality of the software measurement dataset.