Federated Learning Based on Diffusion Model to Cope with Non-IID Data

被引:2
|
作者
Zhao, Zhuang [1 ]
Yang, Feng [1 ,2 ,3 ]
Liang, Guirong [1 ]
机构
[1] Guangxi Univ, Sch Comp Elect & Informat, Nanning 530004, Peoples R China
[2] Guangxi Univ, Guangxi Key Lab Multimedia Commun Network Technol, Nanning 530004, Peoples R China
[3] Guangxi Univ, Key Lab Parallel & Distributed Comp Guangxi Coll, Nanning, Peoples R China
关键词
Federated learning; Non-IID data; Diffusion model; Data augmentation;
D O I
10.1007/978-981-99-8546-3_18
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Federated learning is a distributed machine learning paradigm that allows model training without centralizing sensitive data in a single place. However, non independent and identical distribution (non-IID) data can lead to degraded learning performance in federated learning. Data augmentation schemes have been proposed to address this issue, but they often require sharing clients' original data, which poses privacy risks. To address these challenges, we propose FedDDA, a data augmentation-based federated learning architecture that uses diffusion models to generate data conforming to the global class distribution and alleviate the non-IID data problem. In FedDDA, a diffusion model is trained through federated learning and then used for data augmentation, thus mitigating the degree of non-IID data without disclosing clients' original data. Our experiments on non-IID settings with various configurations show that FedDDA significantly outperforms FedAvg, with up to 43.04% improvement on the Cifar10 dataset and up to 20.05% improvement on the Fashion-MNIST dataset. Additionally, we find that relatively low-quality generated samples that conform to the global class distribution still improve federated learning performance considerably.
引用
收藏
页码:220 / 231
页数:12
相关论文
共 50 条
  • [1] Federated learning on non-IID data: A survey
    Zhu, Hangyu
    Xu, Jinjin
    Liu, Shiqing
    Jin, Yaochu
    NEUROCOMPUTING, 2021, 465 : 371 - 390
  • [2] Adaptive Federated Learning With Non-IID Data
    Zeng, Yan
    Mu, Yuankai
    Yuan, Junfeng
    Teng, Siyuan
    Zhang, Jilin
    Wan, Jian
    Ren, Yongjian
    Zhang, Yunquan
    COMPUTER JOURNAL, 2023, 66 (11): : 2758 - 2772
  • [3] Federated Learning With Taskonomy for Non-IID Data
    Jamali-Rad, Hadi
    Abdizadeh, Mohammad
    Singh, Anuj
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, 34 (11) : 8719 - 8730
  • [4] Federated Learning With Non-IID Data: A Survey
    Lu, Zili
    Pan, Heng
    Dai, Yueyue
    Si, Xueming
    Zhang, Yan
    IEEE INTERNET OF THINGS JOURNAL, 2024, 11 (11): : 19188 - 19209
  • [5] A Survey of Federated Learning on Non-IID Data
    HAN Xuming
    GAO Minghan
    WANG Limin
    HE Zaobo
    WANG Yanze
    ZTECommunications, 2022, 20 (03) : 17 - 26
  • [6] Non-IID Federated Learning
    Cao, Longbing
    IEEE INTELLIGENT SYSTEMS, 2022, 37 (02) : 14 - 15
  • [7] Differentially private federated learning with non-IID data
    Cheng, Shuyan
    Li, Peng
    Wang, Ruchuan
    Xu, He
    COMPUTING, 2024, 106 (07) : 2459 - 2488
  • [8] Fast converging Federated Learning with Non-IID Data
    Naas, Si -Ahmed
    Sigg, Stephan
    2023 IEEE 97TH VEHICULAR TECHNOLOGY CONFERENCE, VTC2023-SPRING, 2023,
  • [9] Adaptive Federated Deep Learning With Non-IID Data
    Zhang, Ze-Hui
    Li, Qing-Dan
    Fu, Yao
    He, Ning-Xin
    Gao, Tie-Gang
    Zidonghua Xuebao/Acta Automatica Sinica, 2023, 49 (12): : 2493 - 2506
  • [10] Federated Dictionary Learning from Non-IID Data
    Gkillas, Alexandros
    Ampeliotis, Dimitris
    Berberidis, Kostas
    2022 IEEE 14TH IMAGE, VIDEO, AND MULTIDIMENSIONAL SIGNAL PROCESSING WORKSHOP (IVMSP), 2022,