Topic correlation model for cross-modal multimedia information retrieval. Issue 4 (November 2016)

• Record Type: Journal Article
• Authors: Qin, Zengchang; Yu, Jing; Cong, Yonghui; Wan, Tao
• Is Part Of: Pattern analysis and applications. Volume 19:Issue 4(2016:Nov.)
• Page Start: 1007
• Page End: 1022
• Publication Date: 2016-11
• Subjects: Cross-modal multimedia retrieval -- Topic correlation model -- Topic models -- Bag-of-features model; Pattern recognition systems -- Periodicals; Pattern perception -- Periodicals; 006.4
• DOI: 10.1007/s10044-015-0478-y
• Languages: English
• ISSNs: 1433-7541

Abstract

Abstract In this paper, we present a simple and effective topic correlation model (TCM) for cross-modal multimedia retrieval by jointly modeling the text and image components in multimedia documents. In this model, the image component is represented by the bag-of-features model based on local scale-invariant feature transform features, meanwhile the text component is described by a topic distribution learned from a latent topic model. Statistical correlations between these two mid-level features are investigated by mapping them into a semantic space. These cross-modality correlations are used to calculate the conditional probabilities of answers in one modality while given query in the other modality. The model is tested on three cross-modal retrieval benchmark problems including Wikipedia documents in both English and Chinese. Experimental results have demonstrated that the new TCM model achieves the best performance compared to recent state-of-the-art cross-modal retrieval models on the given benchmarks.