A privacy preservation framework for feedforward-designed convolutional neural networks

A feedforward-designed convolutional neural network (FF-CNN) is an interpretable neural network with low training complexity. Unlike a neural network trained using backpropagation (BP) algorithms and optimizers (e.g., stochastic gradient descent (SGD) and Adam), a FF-CNN obtains the model parameters in one feed-forward calculation based on two methods of data statistics: subspace approximation with adjusted bias and least squares regression. Currently, models based on FF-CNN training methods have achieved outstanding performance in the fields of image classification and point cloud data processing. In this study, we analyze and verify that there is a risk of user privacy leakage during the training process of FF-CNN and existing privacy-preserving methods for model gradients or loss functions do not apply to FF-CNN models. Therefore, we propose a securely forward-designed convolutional neural network algorithm (SFF-CNN) to protect the privacy and security of data providers for the FF-CNN model. Firstly, we propose the DPSaab algorithm to add the corresponding noise to the one-stage Saab transform in the FF-CNN design for improved protection performance. Secondly, because noise addition brings the risk of model over-fitting and further increases the possibility of privacy leakage, we propose the SJS algorithm to filter the input features of the fully connected model layer. Finally, we theoretically prove that the proposed algorithm satisfies differential privacy and experimentally demonstrate that the proposed algorithm has strong privacy protection. The proposed algorithm outperforms the compared deep learning privacy-preserving algorithms in terms of utility and robustness. (C) 2022 Published by Elsevier Ltd.