Discovering unknown advanced persistent threat using shared features mined by neural networks

Command and control channel(C&C) is used in some cyber attacks to remotely control infected hosts to steal data or conduct espionage. An effective type of C&C detection methods is network flow based. The insight is that network flow is evitable because the hidden malware in the target system has to communicate with the external C&C server to either receive commands or send data. However, existing network flow-based methods face two challenges to efficiently detect C&C of APT(Advanced persistent threat) attacks, i.e., stealth and flexible attack techniques. To combat these two challenges, we design a new network flow-based C&C detection method. Our work is inspired from two observations that different APT attacks share the same intrusion tools and services, and the unknown malware evolves from existing one. Therefore, the malwares of different groups have some shared attributes that are not easy to find, which leads to some hidden shared features in the network flows between the malware and the C&C server in different attacks. Based on this, we propose a method to detect the hidden C&C channel of unknown APT attacks. First, we use deep learning techniques to mine the shared network flow features from the known multi-class attack flows. Later, we use an appropriate classifier to detect the C&C network flow. Finally, we test our method on public available dataset. The experimental results show that our method can achieve up to F1 score of 0.968 when dealing with unknown malicious network flows. This will help discover unknown APT attacks. © 2021 Elsevier B.V.