Towards building data analytics benchmarks for IoT intrusion detection

Data analytics projects span all types of domains and applications. Researchers publish results using certain datasets and classification models. They present results with a summary of the performance metrics of their evaluated classifiers. However, readers and evaluators may not be able to compare results from the different papers for several reasons. One reason is the variations in the classification models and specific settings used in those models; a second reason is the variations of computing resources and environments used to produce those results. A third reason is a variation in the input datasets, preprocessing steps, etc. There is a need for benchmarks or baselines, where researchers can reuse the same benchmark specifications and be able to produce the same reported results. This paper aims to aggregate models from different research papers and apply them using the same classification settings, the same computing resources, and the same datasets and preprocessing steps. The goals are to show a comparable comparison and also to build a benchmark for future assessment and improvements. Four popular IoT security datasets are used in this paper. Different classification models are evaluated and are applied in the same datasets using the same settings. The classifiers results are reported and compared based on several popular performance metrics.