Projected Federated Averaging with Heterogeneous Differential Privacy

被引:20
|
作者
Liu, Junxu [1 ,3 ]
Lou, Jian [2 ,3 ]
Xiong, Li [3 ]
Liu, Jinfei [3 ,4 ,5 ]
Meng, Xiaofeng [1 ]
机构
[1] Renmin Univ China, Beijing, Peoples R China
[2] Xidian Univ, Xian, Peoples R China
[3] Emory Univ, Atlanta, GA 30322 USA
[4] Zhejiang Univ, Hangzhou, Peoples R China
[5] Georgia Tech, Atlanta, GA USA
来源
PROCEEDINGS OF THE VLDB ENDOWMENT | 2021年 / 15卷 / 04期
基金
国家重点研发计划;
关键词
D O I
10.14778/3503585.3503592
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Federated Learning (FL) is a promising framework for multiple clients to learn a joint model without directly sharing the data. In addition to high utility of the joint model, rigorous privacy protection of the data and communication efficiency are important design goals. Many existing efforts achieve rigorous privacy by ensuring differential privacy for intermediate model parameters, however, they assume a uniform privacy parameter for all the clients. In practice, different clients may have different privacy requirements due to varying policies or preferences. In this paper, we focus on explicitly modeling and leveraging the heterogeneous privacy requirements of different clients and study how to optimize utility for the joint model while minimizing communication cost. As differentially private perturbations affect the model utility, a natural idea is to make better use of information submitted by the clients with higher privacy budgets (referred to as "public" clients, and the opposite as "private" clients). The challenge is how to use such information without biasing the joint model. We propose Projected Federated Averaging (PFA), which extracts the top singular subspace of the model updates submitted by "public" clients and utilizes them to project the model updates of "private" clients before aggregating them. We then propose communication-efficient PFA+, which allows "private" clients to upload projected model updates instead of original ones. Our experiments verify the utility boost of both algorithms compared to the baseline methods, whereby PFA+ achieves over 99% uplink communication reduction for "private" clients.
引用
收藏
页码:828 / 840
页数:13
相关论文
共 50 条
  • [1] SECURE FEDERATED AVERAGING ALGORITHM WITH DIFFERENTIAL PRIVACY
    Li, Yiwei
    Chang, Tsung-Hui
    Chi, Chong-Yung
    [J]. PROCEEDINGS OF THE 2020 IEEE 30TH INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING (MLSP), 2020,
  • [2] Efficient federated learning privacy preservation method with heterogeneous differential privacy
    Ling, Jie
    Zheng, Junchang
    Chen, Jiahui
    [J]. COMPUTERS & SECURITY, 2024, 139
  • [3] Federated Data Integration for Heterogeneous Partitions Based on Differential Privacy
    Huang, Jinghao
    Sang, Yingpeng
    Cai, Chaoxin
    Li, Weizheng
    Zhang, Maliang
    [J]. PARALLEL AND DISTRIBUTED COMPUTING, APPLICATIONS AND TECHNOLOGIES, PDCAT 2021, 2022, 13148 : 568 - 575
  • [4] Local differential privacy federated learning based on heterogeneous data multi-privacy mechanism
    Wang, Jie
    Zhang, Zhiju
    Tian, Jing
    Li, Hongtao
    [J]. Computer Networks, 2024, 254
  • [5] Clustered Federated Learning With Adaptive Local Differential Privacy on Heterogeneous IoT Data
    He, Zaobo
    Wang, Lintao
    Cai, Zhipeng
    [J]. IEEE INTERNET OF THINGS JOURNAL, 2024, 11 (01): : 137 - 146
  • [6] A federated learning differential privacy algorithm for non-Gaussian heterogeneous data
    Xinyu Yang
    Weisan Wu
    [J]. Scientific Reports, 13
  • [7] A federated learning differential privacy algorithm for non-Gaussian heterogeneous data
    Yang, Xinyu
    Wu, Weisan
    [J]. SCIENTIFIC REPORTS, 2023, 13 (01)
  • [8] Heterogeneous Differential-Private Federated Learning: Trading Privacy for Utility Truthfully
    Lin, Xi
    Wu, Jun
    Li, Jianhua
    Sang, Chao
    Hu, Shiyan
    Deen, M. Jamal
    [J]. IEEE TRANSACTIONS ON DEPENDABLE AND SECURE COMPUTING, 2023, 20 (06) : 5113 - 5129
  • [9] Personalized Federated Learning With Differential Privacy
    Hu, Rui
    Guo, Yuanxiong
    Li, Hongning
    Pei, Qingqi
    Gong, Yanmin
    [J]. IEEE INTERNET OF THINGS JOURNAL, 2020, 7 (10) : 9530 - 9539
  • [10] Federated f-Differential Privacy
    Zheng, Qinqing
    Chen, Shuxiao
    Long, Qi
    Su, Weijie
    [J]. 24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS), 2021, 130