NetMQ: High-performance In-network Caching for Message Queues with Programmable Switches

Message queues are fundamental components in modern cloud architecture. With the increased data traffic, traditional distributed message queue systems suffer from low performance. We present NetMQ, a new message queue architecture that leverages the power and flexibility of new-generation programmable switches to handle message producing and consuming requests of hot message queues. Due to the limited switch memory, we design a memory layout for multiple cached queues sharing a single register array, and two memory supplementing strategies for different types of queues. To handle dynamic workloads, we design a space-efficient hot queue detector in the data plane, and a heuristic algorithm running in the controller to update the cached topic partition regularly. We implement a NetMQ prototype on Barefoot Tofino switches and commodity servers. Our evaluations show that NetMQ reduces the latency by 3.5-18x, and improves the throughput by 3-3.8x for queues cached in the switch, while incurs negligible overheads for the uncached queues.