mShield: Protecting In-process Sensitive Data Against Vulnerable Third-Party Libraries

Third-party libraries (TPLs) are widely adopted in softwares for integrating special functions (e.g., compression) efficiently. However, as most TPLs are in the same process with the invoker, attackers could exploit memory disclosure vulnerabilities in TPLs to read the sensitive memory data of the victim process. Therefore, once a vulnerability found in a TPL, all softwares with this TPL need to be patched in time, which is impractical. In this paper, we propose a cryptography-based isolation (named mShield) between the data memory of the invoker and TPLs, to prevent TPL vulnerabilities from being exploited to read the invoker's sensitive memory data. mShield performs a user-mode and lightweight memory analysis, figures out the invoker's memory space (including stack, heap, user-defined ones in BSS/data segment), encrypts them before invoking any TPL function, and automatically decrypts them once the function returns, without interrupting the normal execution. mShield performs the encryption/decryption in the trusted environment provided by Intel SGX, which prevents the attacker from reading the cryptographic key, and alerts (i.e., the invoker's decryption fails) in time once the encryption context is tampered with (e.g., by illegal invocations of decryption). We have implemented mShield, and adopted it to protect Nginx against a potentially vulnerable TPL (i.e., zlib). The experiment demonstrates mShield's effectiveness (TPLs fail to read the invoker's plaintext sensitive memory data) and acceptable efficiency (about less than 4x time cost).