ROPGMN: Effective ROP and variants discovery using dynamic feature and graph matching network

Return Oriented Programming (ROP) is one of the most challenging threats to operating systems. Traditional detection and defense techniques for ROP such as stack protection, address randomization, compiler optimization, control flow integrity, and basic block thresholds have certain limitations inaccuracy or efficiency. At the same time, they cannot effectively detect ROP variant attacks, such as COP, COOP, JOP. In this paper, we propose a novel ROP and its variants detection approach that first filters the normal execution flow according to four strategies provided and then adopts Graph Matching Network (GMN) to determine whether there is ROP or its variant attack. Moreover, we developed a prototype named ROPGMN with shared memory to solve cross-language and cross-process problems. Using real-world vulnerable programs and constructed programs with dangerous function calls, we conduct extensive experiments with 6 ROP detectors to evaluate ROPGMN. The experimental results demonstrate the effectiveness of ROPGMN in discovering ROPs and their variant attacks with low performance overhead.