Traffic-Aware Online Network Selection in Heterogeneous Wireless Networks

We focus on the network selection problem in heterogeneous wireless networks. Many traditional approaches select the best network according to quality of service (QoS)-related criteria, which neglects diverse user demands. We aim to select networks maximizing the quality of experience (QoE) of users. When the availability and dynamics of network state information (NSI) are considered, most of the existing approaches cannot make effective selection decisions since they are vulnerable to the uncertainty in NSI. To address this issue, we introduce the idea of online learning for network selection. In this paper, we formulate the network selection problem as a continuous-time multiarmed bandit (CT-MAB) problem. A traffic-aware online network selection (ONES) algorithm is designed to match typical traffic types of users with respective optimal networks in terms of QoE. Moreover, we found that the correlation among multiple traffic network selections can be exploited to improve the learning capability. This motivates us to propose another two more efficient algorithms: the decoupled ONES (D-ONES) algorithm and the virtual multiplexing ONES (VM-ONES) algorithm. Simulation results demonstrate that our ONES algorithms attain around 10% gain in QoE reward rate over nonlearning-based algorithms and learning-based algorithms without QoE considerations.