Secure Enhancement for MQTT Protocol Using Distributed Machine Learning Framework

The Message Queuing Telemetry Transport (MQTT) protocol stands out as one of the foremost and widely recognized messaging protocols in the field. It is often used to transfer and manage data between devices and is extensively employed for applications ranging from smart homes and industrial automation to healthcare and transportation systems. However, it lacks built-in security features, thereby making it vulnerable to many types of attacks such as man-in-the-middle (MitM), buffer overflow, pre-shared key, brute force authentication, malformed data, distributed denial-of-service (DDoS) attacks, and MQTT publish flood attacks. Traditional methods for detecting MQTT attacks, such as deep neural networks (DNNs), k-nearest neighbor (KNN), linear discriminant analysis (LDA), and fuzzy logic, may exist. The increasing prevalence of device connectivity, sensor usage, and environmental scalability become the most challenging aspects that novel detection approaches need to address. This paper presents a new solution that leverages an H2O-based distributed machine learning (ML) framework to improve the security of the MQTT protocol in networks, particularly in IoT environments. The proposed approach leverages the strengths of the H2O algorithm and architecture to enable real-time monitoring and distributed detection and classification of anomalous behavior (deviations from expected activity patterns). By harnessing H2O's algorithms, the identification and timely mitigation of potential security threats are achieved. Various H2O algorithms, including random forests, generalized linear models (GLMs), gradient boosting machine (GBM), XGBoost, and the deep learning (DL) algorithm, have been assessed to determine the most reliable algorithm in terms of detection performance. This study encompasses the development of the proposed algorithm, including implementation details and evaluation results. To assess the proposed model, various evaluation metrics such as mean squared error (MSE), root-mean-square error (RMSE), mean per class error (MCE), and log loss are employed. The results obtained indicate that the H2OXGBoost algorithm outperforms other H2O models in terms of accuracy. This research contributes to the advancement of secure IoT networks and offers a practical approach to enhancing the security of MQTT communication channels through distributed detection and classification techniques.