On Combining Diverse Static Analysis Tools for Web Security: An Empirical Study

Developers frequently rely on free static analysis tools to automatically detect vulnerabilities in the source code of their applications, but it is well-known that the performance of such tools is limited and varies from one software development scenario to another, both in terms of coverage and false positives. Diversity is an obvious direction to take to improve coverage, as different tools usually report distinct vulnerabilities, but this may come with an increase in the number of false alarms. In this paper, we study the problem of combining diverse static analysis tools to detect web vulnerabilities, considering four software development scenarios with different goals and constraints, ranging from low budget to high-end (e.g., business critical) applications. We conducted an experimental campaign with five free static analysis tools to detect vulnerabilities in a workload composed by 134 WordPress plugins. Results clearly show that the best solution depends on the development scenario. Furthermore, in some cases, a single tool performs better than the best combination of tools.