Multi-level just-enough elasticity for MQTT brokers of Internet of Things applications

Applications for the Internet of Things (IoT) are rapidly having an impact on all areas of daily life. Every day, its embedded devices generate loads of data that requires efficient network infrastructure. The integration of lightweight communication protocols such as Message Queuing Telemetry Transport (MQTT) is to send millions of IoT messages back and forth with as few errors as possible. In practice, IoT big data analytic systems are often deployed with highly regarded MQTT solutions to handle huge amounts of dynamic data and achieve scalability. However, these solutions do not adapt well to fluctuations in workload, so they are not elastic yet. This article introduces a novel framework that provides just-enough elasticity for MQTT brokers with multiple levels of virtualization and its implementation using EMQX MQTT broker, Kubernetes container-orchestration system and OpenStack cloud environment. Various experiments based on a real life IoT application are conducted to validate our proposed framework and its elastic functionality.