The Impact Study of Class Imbalance on the Performance of Software Defect Prediction Models

Class imbalance refers to that the number of samples in different classes is unbalanced. In the process of software defect prediction, the performance of traditional prediction models may be affected by the class imbalance problem of datasets. In order to explore the impact of class imbalance on the performance of software defect prediction models, this paper presents an approach to analyzing the impact of class imbalance. First, an algorithm is designed to construct new datasets, which could convert an original imbalanced dataset into a set of new datasets with imbalance ratio increased one by one. Second, different classification models are selected as the defect prediction models to predict on these new constructed datasets respectively. Moreover, AUC metric is used to measure the classification performance of different prediction models. Finally, Coefficient of Variation (C•V) is applied to evaluate the performance stability of each prediction model with class imbalance. The empirical study is conducted on eight typical prediction models. The results show that the performance of three prediction models, C4.5, RIPPER and SMO, are decreased with the increasing of imbalance ratio. However, cost-sensitive learning and ensemble learning could improve their performance and performance stability with class imbalance. Compared with the above three models, the performance of Logistic Regression, Naive Bayes and Random Forest models are more stable. © 2018, Science Press. All right reserved.