Edge-Based Federated Deep Reinforcement Learning for IoT Traffic Management

The wide adoption of large-scale Internet of Things (IoT) systems has led to an unprecedented increase in backhaul (BH) traffic congestion, making it critical to optimize traffic management at the network edge. In IoT systems, the BH network is supported by various backhauling technologies that have different characteristics. Also, the characteristics of the BH links can be sometimes time varying and have an unknown state, due to external factors such as having the resources shared with other systems. It is the responsibility of the edge devices to be able to forward IoT traffic through the unknown-state BH network by selecting the suitable BH link for each collected data flow. To the best of our knowledge, this type of BH selection problem is not addressed in the literature. Therefore, there is a crucial need to develop intelligent approaches enabling edge devices to learn how to deal with unknown-state (partially observable) components of the BH network, which is the primary goal of this article. We propose an edge-based BH selection technique for improving traffic delivery by exploiting multiobjective feedback on delivery performance. The proposed approach relies on the advantage-actor-critic deep reinforcement learning (DRL) methods. Moreover, to improve the DRL training performance in large-scale deployments of distributed IoT systems, federated learning (FL) is applied to enable multiple edge devices to collaborate in training a shared BH selection policy. The proposed federated DRL (F-DRL) approach is able to solve the BH selection problem as verified and demonstrated through extensive simulations.