Malware Obfuscation Detection via Maximal Patterns

Malware obfuscation is defined as a program transformation. It is always used in malware to evade detection from anti-malware software. In this paper, we propose a method to detect malware obfuscation using maximal patterns. Maximal pattern is a subsequence in malware's runtime system call sequence, which frequently appears in program execution, and can be used to describe the program specific behavior. The maximal pattern sequence is extracted from the malware's runtime system calls, and the similarity between two pattern sequences will be measured by evolutionary similarity. Based on the real-world malwares test data, the experiment results have shown that our method can efficiently detect malware obfuscation.