Obfuscated malicious executable scanner

The proliferation of malware (viruses, Trojans, and other malicious code) in recent years has presented a serious threat to individual users, enterprises, and organizations alike. Current static scanning techniques for malware detection have serious limitations; on the other hand, sandbox testing fails to provide a complete satisfactory solution either due to time constraints (e.g., time bombs cannot be detected before its preset time expires). What is making the situation worse is the ease of producing polymorphic (or variants of) computer viruses that are even more complex and difficult than their original versions to detect. In this paper, we propose a new approach for detecting polymorphic malware in the Windows platform. Our approach rests on an analysis based on the Windows API calling sequence that reflects the behaviour of a particular piece of code. The analysis is carried out directly on the PE (pot-table executable) code. It is achieved in two basic steps: construct the API calling sequences for both the known virus and the suspicious code, and then perform a similarity measurement between the two sequences after a sequence realignment operation is done. An alternative technique based on comparing the bags of API calls, and the technique's performance, are also studied. Favourable (in terms of time and accuracy of detection) experimental results are obtained and presented.