Managing the Uncertainty of Bias-Variance Tradeoff in Software Predictive Analytics

The importance of providing accurate estimations of software cost in management life cycle has led to an overabundant pool of prediction candidates exhibiting certain advantages and limitations. Thus, there is an imperative need for well-established principles that will aid the right decision-making regarding the selection of the best candidate. Unfortunately, the choice of the most appropriate estimation technique is not a trivial task, due to the multi-faceted nature of error. Accuracy, bias and variance are notions describing different aspects of predictive power that someone has to take into consideration during the validation process. The main objective of this paper is the utilization of visual analytics for the evaluation of two fundamental ingredients of prediction accuracy: the bias and the variance. Through a bootstrap-based resampling algorithm, we provide an easy-to-interpret way in order to acquire significant knowledge about the quality of a prediction candidate and manage the uncertainty of the estimation process. Ensemble techniques utilizing the advantages of both simple and complex solo methods are possible balancing solutions to the problem of the bias-variance tradeoff.