Do Developers Really Worry About Refactoring Re-test? An Empirical Study of Open-Source Systems

In this paper, we explore the extent to which a set of over 12000 refactorings fell into one of four re-test categories defined by van Deursen and Moonen; the 'least disruptive' of the four categories contains refactorings requiring only minimal re-test. The 'most disruptive' category of refactorings on the other hand requires significant re-test effort. We used multiple versions of three open-source systems to answer one research question: Do developers prefer to undertake refactorings in the least disruptive categories or in the most disruptive? The simple answer is that they prefer to do both. We provide insights into these refactoring patterns across the systems and highlight a fundamental weakness with software metrics trying to capture the refactoring process.