Privacy-Preservation in Online Distributed Dual Averaging Optimization

Distributed optimization allows a separate set of data owners to collaboratively optimize a learning model, wherein information exchange between agents is usually explicit, and hence it is easy to cause the leakage of sensitive information. In order to address the critical issue of data privacy, a distributed online optimization algorithm with privacy protection is proposed in this paper. With the weight between two connected agents being decomposed into weight pairs, the homomorphic encryption mechanism (Paillier Cryptosystem) and the online distributed dual averaging algorithm are combined to propose an online distributed dual averaging privacy-protection algorithm. We prove that the sublinear regret bound and privacy protection can be guaranteed for the strongly connected undirected network. Finally, theoretical analysis and numerical simulations show that the adversaries can not steal the sensitive information of neighboring agents when collecting the multi-step intermediate information, therefore the algorithm can effectively protect the agents' privacy of the network.