Joint Edge Association and Aggregation Frequency for Energy-Efficient Hierarchical Federated Learning by Deep Reinforcement Learning

Hierarchical Federated Learning (HFL) has been proposed to achieve larger-scale model training and more efficient communications compared to conventional Federated Learning (FL). However, both inappropriate edge association strategy and aggregation frequency may consume massive energy in users with poor channel conditions or degrade the HFL convergence performance due to Non Independent and Identical Distribution (NIID) data, which is challenging to energy-limited users. Motivated by this, a dynamically joint edge association and aggregation frequency optimization problem is proposed from the perspective of minimizing long-term energy consumption. By incorporating the communication model and convergence analysis, the problem can be formulated to strike a balance between HFL convergence rate and energy consumed by all users within one global communication round. Then, a Deep Reinforcement Learning (DRL) agent is designed to approximate the optimal solution. Simulation results verify the convergence analysis and the proposed DRL-assisted joint strategy can consume the least energy while reaching the required target model accuracy compared to other benchmarks.