A Novel Incentive Mechanism for Federated Learning over Wireless Communications

This paper studies a federated learning system over wireless communications, where a parameter server shares a global model trained by distributed devices. Due to limited communication resources, not all devices can participate in the training process. To encourage suitable devices to participate, this paper proposes a novel incentive mechanism, where the parameter server assigns rewards to the devices, and the devices make participation decisions to maximize their overall profit based on the obtained rewards and their energy costs. Based on the interaction between the parameter server and the devices, the proposed incentive mechanism is formulated as a bilevel optimization problem (BOP), in which the upper level optimizes reward factors for the parameter server and the lower level makes participation decisions for the devices. Note that each device needs to make an independent participation decision due to limited communication resources and privacy concerns. To solve this BOP, a bilevel optimization approach called BIMFL is proposed. BIMFL adopts multi-agent reinforcement learning (MARL) to make independent participation decisions with local information at the lower level, and introduces multi-agent meta-reinforcement learning to accelerate the training by incorporating meta-learning into MARL. Moreover, BIMFL utilizes covariance matrix adaptation evolutionary strategy to optimize reward factors at the upper level. The effectiveness of BIMFL is demonstrated on different datasets using multilayer perceptron and convolutional neural networks. IEEE