Differential Privacy-preserving Distributed Machine Learning

Distributed machine learning (DML) has received widespread attentions, where a shared prediction model is collaboratively learned by multiple servers. However, since the data used for model training often contains users' sensitive information, DML faces potential risks of privacy disclosure. Particularly, when servers are untrustworthy, it is critical while challenging to guarantee users to obtain privacy preservation that is self-controllable and does not weaken in strength during the whole DML process. In this paper, we propose a privacy-preserving solution for DML, where privacy protection is achieved through data randomization at the users' side and a modified alternating direction method of multipliers (ADMM) algorithm is designed for servers to mitigate the effect of data perturbation. We prove that this solution provides differential privacy guarantee and preserves the convergence property of a general ADMM paradigm. Also, we provide extensive theoretical analysis about the performance of the trained model. Numerical experiments using standard classification datasets are finally conducted to validate the theoretical results.