A Novel Distributed Machine Learning Framework for Semi-Supervised Detection of Botnet Attacks

In today's Internet world where everything is interconnected, the misuse of the shared communication channels and the service providers by the malicious users requires all time monitoring. Amongst the various methods being adopted by network attackers, like Distributed Denial of Service (DDoS) attacks, spams, phishing attacks, etc. botnet threats are increasing day-by-day. Detecting botnet attacks is a challenging task. Firstly, botnets are difficult to detect because of stealthy nature of Command & Control protocol. Secondly, different types of bots have varied characteristics and combined with large size of the network traffic their detection becomes a very challenging task. Lastly, network traffic is unlabeled and classification techniques like decision trees cannot be used directly. Moreover with the success of distributed frameworks like Hadoop and Apache Spark it is feasible to handle very large data. In this paper we have used distributed framework to apply semi-supervised machine learning techniques of KMeans clustering for labeling a large dataset and decision trees as classifiers. High accuracy was achieved in prediction of the classes. Novelty of our work is labeling of unlabeled network traffic and classification using efficient distributed frameworks.