Effectiveness of code reading and functional testing with event-driven object-oriented software

This chapter is concerned with experimental comparisons of code reading and functional testing (including fault identification) of concurrent event-driven Java software. Our initial idea was that functional-testing is more effective than code reading with respect to concurrent event-driven OO software. A controlled experiment was initially conducted with sophomore students (inexperienced subjects). Subsequently, it was replicated with some changes with junior and senior students (moderately experienced subjects). We also conducted a further replication with Master students, which is not considered in this Chapter. The experiment goal was studied from different perspectives, including effect of techniques on the different types of faults. Results can be overviewed as the following: 1) Concerning the initial, basic experiment: with inexperienced subjects and a strict interval of inspecting time of two hours, there was no statistically significant difference between the techniques under consideration; subjects performance indicator was 62% for code reading and 75% for functional testing. 2) Concerning the (first) replication: with moderately expert subjects, again a strict interval of inspecting time of two hours, and more than twice number of seeded faults, there was no statistically significant difference between the techniques; subjects performance indicator was 100% for code reading and 92% for functional testing; subjects performance indicator shows that more experienced subjects were asking for more inspecting time; however, functional testing performed much better than in the basic experiment. Computation faults were the most detectable for code reading while control faults were the most detectable for functional testing. Moreover, moderately expert subjects were more effective than inexperienced ones in detecting interface and event types of faults. Furthermore moderately expert functional testers detected many preexistent (non-seeded) faults, while both inexperienced subjects, and moderately experienced code readers could not detect non-seeded faults.