FL-PATE: Differentially Private Federated Learning with Knowledge Transfer

被引:6
|
作者
Pan, Yanghe [1 ]
Ni, Jianbing [2 ]
Su, Zhou [1 ]
机构
[1] Xi An Jiao Tong Univ, Sch Cyber Sci & Engn, Xian 710000, Peoples R China
[2] Queens Univ, Dept Elect & Comp Engn, Kingston, ON K7L 3N6, Canada
关键词
Differential Privacy; Knowledge Transfer; Federated Learning; Privacy Preservation;
D O I
10.1109/GLOBECOM46510.2021.9685079
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Federated learning provides a solution for data privacy protection, while enabling training over the local data samples, without exchanging them. However, it is far from practical and secure because data privacy is still vulnerable due to the well-studied attacks, e.g., membership inference attacks and model inversion attacks. In this paper, to further prevent data leakage against these attacks, we propose FL-PATE, a differentially private federated learning framework with knowledge transfer. Specifically, participants with sensitive data are grouped to train teacher models under federated learning settings, and the knowledge of teacher models is transferred to a publicly accessible student model for prediction via aggregating teacher models' outputs of public datasets. A modified client-level differential privacy mechanism is used to guarantee each participant's data privacy during the corresponding teacher model's training process. The proposed framework preserves participant's privacy against membership inference attacks and the differential privacy cost is fixed. The privacy analysis and experiments demonstrate that trained teacher and student models have an excellent performance in accuracy and robustness theoretically and empirically.
引用
收藏
页数:6
相关论文
共 50 条
  • [1] Differentially private knowledge transfer for federated learning
    Qi, Tao
    Wu, Fangzhao
    Wu, Chuhan
    He, Liang
    Huang, Yongfeng
    Xie, Xing
    [J]. NATURE COMMUNICATIONS, 2023, 14 (01)
  • [2] Differentially private knowledge transfer for federated learning
    Tao Qi
    Fangzhao Wu
    Chuhan Wu
    Liang He
    Yongfeng Huang
    Xing Xie
    [J]. Nature Communications, 14
  • [3] Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise
    Fu, Jie
    Chen, Zhili
    Han, Xiao
    [J]. 2022 IEEE INTERNATIONAL CONFERENCE ON TRUST, SECURITY AND PRIVACY IN COMPUTING AND COMMUNICATIONS, TRUSTCOM, 2022, : 656 - 663
  • [4] DP-FL: a novel differentially private federated learning framework for the unbalanced data
    Xixi Huang
    Ye Ding
    Zoe L. Jiang
    Shuhan Qi
    Xuan Wang
    Qing Liao
    [J]. World Wide Web, 2020, 23 : 2529 - 2545
  • [5] FDP-FL: differentially private federated learning with flexible privacy budget allocation
    Qian, Wenjun
    Shen, Qingni
    Chen, Xiaoyi
    Li, Cong
    Fang, Yuejian
    Wu, Zhonghai
    [J]. COMPUTER JOURNAL, 2024,
  • [6] DP-FL: a novel differentially private federated learning framework for the unbalanced data
    Huang, Xixi
    Ding, Ye
    Jiang, Zoe L.
    Qi, Shuhan
    Wang, Xuan
    Liao, Qing
    [J]. WORLD WIDE WEB-INTERNET AND WEB INFORMATION SYSTEMS, 2020, 23 (04): : 2529 - 2545
  • [7] A Knowledge Transfer Framework for Differentially Private Sparse Learning
    Wang, Lingxiao
    Gu, Quanquan
    [J]. THIRTY-FOURTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THE THIRTY-SECOND INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE AND THE TENTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2020, 34 : 6235 - 6242
  • [8] Differentially Private Federated Knowledge Graphs Embedding
    Peng, Hao
    Li, Haoran
    Song, Yangqiu
    Zheng, Vincent
    Li, Jianxin
    [J]. PROCEEDINGS OF THE 30TH ACM INTERNATIONAL CONFERENCE ON INFORMATION & KNOWLEDGE MANAGEMENT, CIKM 2021, 2021, : 1416 - 1425
  • [9] SignDS-FL: Local Differentially Private Federated Learning with Sign-based Dimension Selection
    Jiang, Xue
    Zhou, Xuebing
    Grossklags, Jens
    [J]. ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY, 2022, 13 (05)
  • [10] Differentially Private Federated Learning with Functional Mechanism
    Cao, Shi-Xiang
    Chen, Chao-Meng
    Tang, Peng
    Su, Sen
    [J]. Jisuanji Xuebao/Chinese Journal of Computers, 2023, 46 (10): : 2178 - 2195