A privacy preservation framework for feedforward-designed convolutional neural networks. (November 2022)

• Record Type: Journal Article
• Authors: Li, De; Wang, Jinyan; Li, Qiyu; Hu, Yuhang; Li, Xianxian
• Is Part Of: Neural networks. Volume 155(2022)
• Page Start: 14
• Page End: 27
• Publication Date: 2022-11
• Subjects: Differential privacy -- Convolutional neural networks -- Feedforward-designed -- Feature selection -- Over-fitting; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2022.08.005
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: A feedforward-designed convolutional neural network (FF-CNN) is an interpretable neural network with low training complexity. Unlike a neural network trained using backpropagation (BP) algorithms and optimizers (e.g., stochastic gradient descent (SGD) and Adam), a FF-CNN obtains the model parameters in one feed-forward calculation based on two methods of data statistics: subspace approximation with adjusted bias and least squares regression. Currently, models based on FF-CNN training methods have achieved outstanding performance in the fields of image classification and point cloud data processing. In this study, we analyze and verify that there is a risk of user privacy leakage during the training process of FF-CNN and existing privacy-preserving methods for model gradients or loss functions do not apply to FF-CNN models. Therefore, we propose a securely forward-designed convolutional neural network algorithm (SFF-CNN) to protect the privacy and security of data providers for the FF-CNN model. Firstly, we propose the DPSaab algorithm to add the corresponding noise to the one-stage Saab transform in the FF-CNN design for improved protection performance. Secondly, because noise addition brings the risk of model over-fitting and further increases the possibility of privacy leakage, we propose the SJS algorithm to filter the input features of the fully connected model layer. Finally, we theoretically prove that the proposed algorithm satisfies differential privacy and experimentally demonstrate that the proposed algorithm has strong privacy protection. The proposed algorithm outperforms the compared deep learning privacy-preserving algorithms in terms of utility and robustness. Highlights: The privacy leakage problem of FF-CNN is analyzed and verified. The privacy budget scheme of eigenvalue proportion is used in DPSaab algorithm. Label smoothing technique to guide a soft feature selection algorithm. Experimental results show the utility and efficiency of the proposed algorithms.