Cross-Modal Search for Social Networks via Adversarial Learning. (11th July 2020)

• Record Type: Journal Article
• Authors: Zhou, Nan; Du, Junping; Xue, Zhe; Liu, Chong; Li, Jinxuan
• Other Names: Köker Raşit Academic Editor.
• Is Part Of: Computational intelligence and neuroscience. Volume 2020(2020)
• Publication Date: 2020-07-11
• Subjects: Neurosciences -- Data processing -- Periodicals; Computational intelligence -- Periodicals; Computational neuroscience -- Periodicals; 612.80285
• DOI: 10.1155/2020/7834953
• Languages: English
• ISSNs: 1687-5265

Abstract

Abstract : Cross-modal search has become a research hotspot in the recent years. In contrast to traditional cross-modal search, social network cross-modal information search is restricted by data quality for arbitrary text and low-resolution visual features. In addition, the semantic sparseness of cross-modal data from social networks results in the text and visual modalities misleading each other. In this paper, we propose a cross-modal search method for social network data that capitalizes on adversarial learning (cross-modal search with adversarial learning: CMSAL). We adopt self-attention-based neural networks to generate modality-oriented representations for further intermodal correlation learning. A search module is implemented based on adversarial learning, through which the discriminator is designed to measure the distribution of generated features from intramodal and intramodal perspectives. Experiments on real-word datasets from Sina Weibo and Wikipedia, which have similar properties to social networks, show that the proposed method outperforms the state-of-the-art cross-modal search methods.