Threats to validity in search-based predictive modelling for software engineering

A number of studies in the literature have developed effective models to address prediction tasks related to a software product such as estimating its development effort, or its change/defect proneness. These predictions are critical as they help in identifying weak areas of a software product and thus guide software project managers in effective allocation of project resources to these weak parts. Such practices assure good quality software products. Recently, the use of search-based approaches (SBAs) for developing software prediction models (SPMs) has been successfully explored by a number of researchers. However, in order to develop effective and practical SPMs it is imperative to analyse various sources of threats. This study extensively reviews 93 primary studies, which use SBAs for developing SPMs of four commonly used software attributes (effort, defect-proneness, maintainability and change-proneness) in order to discuss and identify the various sources of threats while using these approaches for SPMs. The study also lists various actions that may be taken in order to minimise these threats. Furthermore, best practice examples in literature and the year-wise trends of threats indicating the most common threats missed by researchers are provided to help academicians and practitioners in designing effective studies for developing SPMs using SBAs.