Test item prioritizing metrics for selective software testing

In order to respond to the active market's needs for software with various new functions, the system testing must be completed within a limited period. Additionally, important faults, which are closely related to essential functions for users or the target system, have to be removed, preferably in system testing. Many techniques have been proposed to date for effective software testing. Among them, selective software testing is one of the most cost effective techniques. However, most of the previous techniques cannot be applied to short-term development and initial development of software with various new functions because much cost is needed for their testing preparation. In this paper, we propose a new method for selective system testing in which priorities assigned to functions play an essential role in the execution of testing. The priorities are determined based on the evaluation results of three metrics for functions: the frequency of use, the complexity of use scenario, and the fault impact to users. Detailed testing instructions are assigned to test items with high priority, and short and ordinal instructions are assigned to those with low priority. The difference in the volume of testing instruction controls the effort of checking test items. As a result of experimental application to actual software testing in a certain company, we have confirmed that the proposed selective system testing can detect both fatal faults related to key functions and critical faults for the system.