CrashTalk: Automated Generation of Precise, Human Readable, Descriptions of Software Security Bugs

Understanding the cause, consequences, and severity of a security bug are critical facets of the overall bug triaging and remediation process. Unfortunately, diagnosing failures is often a laborious process that requires developers to expend significant time and effort. While solutions have been proposed to help expedite the process of pinpointing the cause of a security bug, few proposals provide an explanation along with a diagnosis to make the bug discovery and triaging process less taxing. Moreover, even in cases where descriptions are provided, they are not guided by classification models that support precise descriptions of the flaw. We present an approach that uses static and dynamic analysis techniques to automatically infer the cause and consequences of a software crash and present diagnostic information following NIST's recently released Bugs Framework taxonomy. Specifically, starting from a crash, we generate a detailed and accessible English description of the failure along with its weakness types and severity, thereby easing the burden on developers and security analysts alike. To evaluate the effectiveness of our approach, we compare our ability to find fault locations and generate explanations compared to that of professional software developers by using a benchmark specifically designed to assist with realistic evaluation of tools in software engineering. In addition, using 33 real-world vulnerabilities we collected, we show that our approach correctly diagnoses over 94% of the failures and, in some cases, generates weakness types that are more specific than those that were originally assigned by the submitter or National Vulnerability Database analysts. We also generate initial vulnerability scores that can be used by project managers to assist with prioritizing bug fixes. On average, the overall process takes just over a minute, which is orders of magnitude faster than what professional developers can do.