Privacy-Preserving Convex Factorization Machine

Factorization Machine (FM) significantly improves the prediction accuracy of the linear models due to its powerful capability of feature combination and has been widely used in various systems, i.e., in recommendation systems and online advertisement. Convex FM further solves the bad local minima issue, which is incurred by the non-convex optimization problem and achieves better prediction performance. However, convex FM has a serious privacy issue since the learning process may reveal the sensitive information of training data. In order to mitigate the information leakage while achieving good prediction performance, we design a privacy-preserving convex Factorization Machine algorithmic framework that satisfies Renyi Differential Privacy. Our algorithmic framework called Out-CFM introduces noises to the learned optimizer at each round and delicately leverages the privacy amplification theorem to reduce the perturbations. Moreover, we utilize a modified Frank-Wolfe method to eschew the expensive projection operation in the optimization process. In addition, we develop two classical learning tasks, i.e., regression and classification tasks, in the Out-CFM framework to enable privacy-preserving learning. We provide the theoretical proof of the privacy guarantee for the algorithmic framework and derive an RDP-based analytical model for the utility analysis. Extensive experiments are performed on the real datasets for both regression and classification tasks to evaluate the performance of our proposals, which validate our theoretical analysis and demonstrate that our proposed methods outperform the baseline methods.