Delay-Optimal Scheduling for Heavy-Tailed and Light-Tailed Flows via Reinforcement Learning

We consider a delay-optimal scheduling problem in a queueing system with a mix of heavy-tailed and light-tailed flows. A light-tailed flow requires more stringent quality of services (QoSs) than a heavy-tailed flow. However, the arrival process of a heavy-tailed flow is far more bursty than that of a light-tailed flow. In addition, flows having a similar tail distribution also require distinct QoSs. This is a NP-hard problem in general. We propose a scheduling scheme that consists of two separate and parallel algorithms, including dynamic-weight-earliest-deadline-first (DWEDF) and reinforcement learning (RL), called DWEDF-RL, to address it. Specifically, we provide delay-bound-based fairness to flows having similar tail distributions in intra-queue buffering process with DWEDF. Inter-queue scheduling process further maximizes the QoS provisioning efficiency by dynamically prioritizing light-tailed flows according to network environments and QoS requirements via reinforcement learning. The effectiveness of the proposal in QoS provisioning has been demonstrated through simulation results.