Thompson Sampling-Based Heterogeneous Network Selection Considering Stochastic Geometry Analysis

We propose a sophisticated network selection scheme based on multi-armed bandits and stochastic geometry for heterogeneous cellular networks. In the system model, a user seeking the best network tries to estimate the density of active interferers for every network through the repeated observation of signal-to-interference power ratio (SIR), which shows the randomness induced by randomized interference sources and fading effects. The purpose of this study is to enable the user to identify the network with the lowest density of active interferers while considering the communication quality during exploration. In order to resolve the trade-off between getting more observations on uncertain networks and using a network that seems better so far, we employ a bandit algorithm called Thompson sampling (TS), which is known for its empirical effectiveness. We take two ideas into consideration to enhance TS. First, noticing that the statistical SIR model given by stochastic geometry is useful for capturing the relationship between observed SIR and density of active interferers, we propose to incorporate the statistical model into TS. Second, TS requires us to sample from the posterior distribution of the density parameter for each network, while the distribution obtained through stochastic geometry is much more complicated to generate samples than well-known distribution; we reveal that such a sampling process is achieved with the help of the Markov chain Monte Carlo method. The simulation results show that the proposed method enables a user to find the best network more efficiently than well-known bandit algorithms such as an epsilon-greedy strategy.