Class-Balanced Federated Learning Based on Data Generation

Modern terminal devices such as mobile phones and wearable devices produce massive amounts of data every day, but these data often involve sensitive privacy and thus cannot be directly disclosed and used. To solve this problem, Federated Learning (FL) has been developed as an important machine learning framework under privacy protection, which allows extensive terminal devices/clients to collaboratively learn a superior global model, without sharing the private data on the clients. However, in practical application, there are still two underlying limitations to existing FL mechanism. First, the global model needs to consider the data on multiple clients, but each client usually contains only partial classes of data and the data amount of different classes is severely imbalanced, making it difficult to train the global model.Specifically, most data on the client belong to a few classes, while other classes have few or no data. As a result, the trained local models tend to overfit the data on the clients and achieve poor performance on global data, which severely affects the training of the global model. Second, the data distribution is extremely different across the clients, which causes the trained models on each client to be quite different, making it hard to derive a promising global model. In fact, the training data on each client usually come from the usage of the terminal device by a particular user. Due to the differences in the functions of the terminal devices and the usage habits of users, different clients often produce different classes of data, leading to extremely different class distribution across the data on the clients. Consequently, there will be huge differences among the local models trained on such distribution, making it difficult to obtain a superior global model through the traditional approach of element-wise weighted averaging model parameters. To reduce the impact of class imbalance and distribution differences, in this paper, we propose a novel Class-Balanced Federated Learning (CBFL) method based on data generation, which aims to produce a class-balanced data set suitable for the training of global model for each client through data generation technique. To this end, CBFL designs a class distribution equalizer that consists of a class-balanced sampler and a data generator. First, the class-balanced sampler samples those classes that have insufficient data on the client with a higher sample probability. Then, the data generator generates corresponding dummy data according to the classes sampled by the class-balanced sampler. Finally, each client combines its original data and the generated data to produce a class-balanced data set for training. In this way, the performance of each local model can be greatly improved and the differences among local models arc highly reduced, which contributes to obtaining a promising global model. Moreover, to obtain high-quality generated data, we exploit global data distribution information from the global model to train the data generator. Extensive experiments on four benchmark datasets demonstrate the superior performance of the proposed method over existing methods. For example, the ResNet20 model trained on CIFAR-100 dataset by the proposed CBFL outperforms existing methods by 5. 82% in terms of accuracy. © 2023 Science Press. All rights reserved.