Optimizing anomaly-based attack detection using classification machine learning

One of the significant aspects of our digital world is that data are literally everywhere, and it is increasing. On the other hand, the number of cyberattacks aiming to seize this data and use it illegally is increasing at an exponential rate, and this is the challenge. Therefore, intrusion detection systems (IDS) have attracted considerable interest from researchers and industries. In this regard, machine learning (ML) techniques are playing a pivotal role as they put the responsibility of analyzing enormous amounts of data, finding patterns, classifying intrusions, and solving issues on computers instead of humans. This paper implements two separate classification layers of ML-based algorithms with the recently published NF-UQ-NIDS-v2 dataset, preprocessing two volumes of sample records (100 k and 10 million), utilizing MinMaxScaler, LabelEncoder, selecting superlative features by recursive feature elimination, normalizing the data, and optimizing hyper-parameters for classical algorithms and neural networks. With a small dataset volume, the results of the classical algorithms layer show high detection accuracy rates for support vector (98.26%), decision tree (98.78%), random forest (99.07%), K-nearest neighbors (98.16%), CatBoost (99.04%), and gradient boosting (98.80%). In addition, the layer of neural network algorithms has proven to be a very powerful technology when using deep learning, particularly due to its unique ability to effectively handle enormous amounts of data and detect hidden correlations and patterns; it showed high detection results, which were (98.87%) for long short-term memory and (98.56%) for convolutional neural networks.