A Q-learning-based Multipath Scheduler for Data Transmission Optimization in Heterogeneous Wireless Networks

In the era of 5G and beyond, mobile devices usually can access several heterogeneous wireless networks (e.g., Wi-Fi and 5G). To simultaneously and efficiently utilize the accessible network resources, multipath transport protocols, such as MPTCP and MPQUIC, have shown much potential. In these protocols, scheduling is one of the critical processes to ensure the performance of the multipath transmission. Although there have been many proposed multipath schedulers in the literature, they have not performed well in heterogeneous networks, especially when the network conditions vary (i.e., dynamicity). In this paper, we propose a novel Q-learning-based Multipath scheduler for data transmission optimization (Q-SAT), aiming to bypass the existing limitation. By leveraging the self-learning ability of reinforcement learning, Q-SAT can instantly observe environmental changes and deploy appropriate path selection to optimize data transmission time. As a result, Q-SAT efficiently schedules multipath communication in heterogeneous wireless networks with different dynamicity levels. We have implemented Q-SAT with MPQUIC and extensively evaluated Q-SAT in an emulated environment and a real network. The evaluation results show that Q-SAT improves the data transmission time by at least 10% in the emulation and 26% in the actual deployment compared to the state-of-the-art schedulers.