Federated Reinforcement Learning with Adaptive Training Times for Edge Caching

To relieve the backhaul link stress and reduce the content acquisition delay, mobile edge caching has become one of the promising approaches. In this paper, a novel federated reinforcement learning (FRL) method with adaptive training times is proposed for edge caching. Through a new federated learning process with the asynchronous model training process and synchronous global aggregation process, the proposed FRL-based edge caching algorithm mitigates the performance degradation brought by the non-identically and independently distributed (non-i.i.d.) characteristics of content popularity among edge nodes. The theoretical bound of the loss function difference is analyzed in the paper, based on which the training times adaption mechanism is proposed to deal with the tradeoff between local training and global aggregation for each edge node in the federation. Numerical simulations have verified that the proposed FRL-based edge caching method outperforms other baseline methods in terms of the caching benefit, the cache hit ratio and the convergence speed.