A multimodal differential privacy framework based on fusion representation learning. Issue 1 (31st December 2022)

• Record Type: Journal Article
• Authors: Cai, Chaoxin; Sang, Yingpeng; Tian, Hui
• Is Part Of: Connection science. Volume 34:Issue 1(2022)
• Page Start: 2219
• Page End: 2239
• Publication Date: 2022-12-31
• Subjects: Differential privacy -- Fourier transform -- binary encoding -- multimodal fusion -- low-rank decomposition -- representation learning; Neural computers -- Periodicals; Artificial intelligence -- Periodicals; Cognitive science -- Periodicals; Connectionism -- Periodicals; 006.3
• DOI: 10.1080/09540091.2022.2111406
• Languages: English
• ISSNs: 0954-0091

Abstract

Abstract : Differential privacy mechanisms vary in modalities, and there have been many methods implementing differential privacy on unimodal data. Few studies focus on unifying them to protect multimodal data, though privacy protection of multimodal data is of great significance. In our work, we propose a multimodal differential privacy protection framework. Firstly, we use multimodal representation learning to fuse different modalities and map them to the same subspace. Then based on this representation, we use the Local Differential Privacy (LDP) mechanism to protect data. We propose two protection methods for low-dimensional and high-dimensional fusion tensors respectively. The former is based on Binary Encoding, and the latter is based on multi-dimensional Fourier Transform. To the best of our knowledge, we are the first to propose LDP-based methods for the representation learning of multimodal fusion. Experimental results demonstrate the flexibility of our framework where both approaches show efficient performance as well as high data utility.