Ransomware early detection using deep reinforcement learning on portable executable header

With the increasing number of ransomware attacks on critical infrastructures, there is an urgent need to develop effective systems that can detect ransomware early. In order to achieve this objective, many detection solutions rely on machine learning to analyze the features of ransomware samples. However, these solutions often need to execute ransomware to extract sufficient features, which can increase the risk of virus infection. This paper proposes a novel static analysis framework based on the portable executable header, which utilizes deep reinforcement learning for early detection of ransomware. The framework leverages the portable executable header (PE header) from executable files as the key feature to identify ransomware. Moreover, the framework can learn the features of ransomware by using deep reinforcement learning. This is achieved through the interaction between the agent and the environment, and then the samples are segmented into ransomware and benign categories by taking actions. The proposed framework achieves rapid detection speeds without running ransomware samples by employing a lightweight network and the portable executable header from the raw byte files. To the best of our knowledge, we are the first to exploit deep reinforcement learning on the PE header for ransomware early detection. Our experiments on two real-world datasets reveal that the proposed framework can rapidly detect unseen ransomware.