Quality-Oriented Federated Learning on the Fly

With federated learning, a large number of edge devices are engaged to train a global model collaboratively using their local private data. To train a high-quality global model, however, recent studies recognized that the quality of model contributions from local training on different edge devices are substantially different. Existing mechanisms for quantifying such model quality are intuitively based on the training loss or model parameters, and fail to capture the effect of highly variable data and heterogeneous resources available on participating edge devices. In this article, we propose a new aggregation mechanism that uses deep reinforcement learning to dynamically evaluate the quality of model updates, with accommodations for both data and device heterogeneity as the training process progresses. By dynamically mapping the quality of local models to their importance during model aggregation, the global training process is able to converge toward the direction of better effectiveness and generalization. We show that our proposed mechanism outperforms its state-of-the-art counterparts, achieving faster convergence and more stable learning progress. Further, the LSTM-TD3 architecture and state representation design in our mechanism allows it to adapt to various unseen federated learning environments with an arbitrary number of local updates.