Analyzing Implementation-Based SSL/TLS Vulnerabilities with Binary Semantics Analysis

SSL/TLS are cryptographic protocols created to protect the security and privacy over computer network communication. As a critical security infrastructure on the internet, it has been widely used for decades in various network related applications, such as HTTPs, SMTPs, FTPs, and so on. Although it is designed to "protect" the network communication, it also has some security concerns. In this paper, we present the feasibility of analyzing implementation-based SSL/TLS vulnerabilities with binary semantics analysis. We use a basic-blocks-sequence based binary semantics comparison method to conduct vulnerability analysis on SSL/TLS vulnerabilities. We abstract a vulnerability execution trace as a "signature". By comparing the semantic similarity of a target program's execution trace and a vulnerability's "signature", we are able to detect whether the target program contains the vulnerability or not. We analyzed the well-known Heartbleed vulnerability and other implementation based vulnerabilities in representative network applications which use two popular SSL/TLS libraries, OpenSSL and mbedTLS. The evaluation result shows that our basic-blocks-sequence based binary semantics comparison method is effective on analyzing the existence of various implementation based SSL/TLS vulnerabilities.