Using Machine Learning to Prioritize Automated Testing in an Agile Environment

Automated software testing is an integral part of most Agile methodologies. In the case of the Scrum Agile methodology, the definition of done includes the completion of tests. As a software project matures, however, the number of tests increases to such a point that the time required to run all the tests often hinders the speed in which artifacts can be deployed. This paper describes a technique of using machine learning to help prioritize automated testing to ensure that tests which have a higher probability of failing are executed early in the test run giving the programmers an early indication of problems. In order to do this, various metrics are collected about the software under test including Cyclomatic values, Halstead-based values, and Chidamber-Kemere values. In addition, the historical commit messages from the source code control system is accessed to see if there had been defects in the various source classes previously. From these two inputs, a data file can be created which contains various metrics and whether or not there had been defects in these source files previously. This data file can then be sent to Weka to create a decision tree indicating which measurements indicate potential defects. The model created by Weka can then then be used in future to attempt to predict where defects might be in the source files and then prioritize testing appropriately.