Security of Open Source Web Applications

In an empirical study of fourteen widely used open source PHP web applications, we found that the vulnerability density of the aggregate code base decreased from 8.88 vulnerabilities/KLOC to 3.30 from Summer 2006 to Summer 2008. Individual web applications varied widely, with vulnerability densities ranging from 0 to 121.4 at the beginning of the study. While the total number of security problems decreased, vulnerability density increased in eight of the fourteen applications over the analysis period. We developed a security resources indicator metric, which we found to be strongly correlated (p = 0.67, p < 0.05) with change in vulnerability density over time. Traditional software metrics, such as code size, cyclomatic complexity, nesting complexity, and churn, had significant (p < 0.05) but much smaller correlations (p = 0.31 at best) with vulnerability density. Vulnerability density was measured using the Fortify Source Code Analyzer static analysis tool.