Machine Learning Classification of Obfuscation using Image Visualization

As the need for new techniques to analyze obfuscated software has grown, recent work has shown the ability to analyze programs via machine learning in order to perform automated metadata recovery. Often these techniques really on disassembly or other means of direct code analysis. We showcase an approach combining code visualization and image analysis via convolutional neural networks capable of statically classifying obfuscation transformations. By first turning samples into gray scale images, we are able to analyze the structure and side effects of transformations used in the software with no heavy code analysis or feature preparation. With experimental results samples produced with the Tigress and OLLVM obfuscators, our models are capable of labeling transformations with F1-scores between 90% and 100% across all tests. We showcase our approach via models designed as both a binary classification problem as well as a multi label and multi output problem. We retain high performance even in the presence of multiple transformations in a file.