A Digital Twin Approach for Self-optimization of Mobile Networks

Most of the methods in operators' current 5G networks use expert knowledge assisted by machine learning algorithms to generate optimization decisions. However, these methods are inadaptive to the dynamic changes of high-dimensional network states, thus the result is often suboptimal. Reinforcement learning can better cope with high-dimensional network state space and parameter space. However, when applied to real network, challenges arise such as difficult to obtain data samples, time-consuming and risky to explore real networks during model training. To solve these problems, this paper proposes a combined approach of expert knowledge, reinforcement learning and digital twin for the self-optimization of mobile networks. By constructing a digital twin of the current network, the future network state is predicted based on which optimization decisions are generated by expert knowledge and reinforcement learning respectively, and then input into the digital twin. Digital twin simulates their rewards and decides a final action for execution. Simulation results have confirmed that the proposed scheme can achieve higher rewards than either expert knowledge or reinforcement learning, and can avoid negative impact on real network performance. This paper also describes several potential application scenarios for the proposed approach in 6G networks and discusses key issues for future research.