AMM: Towards adaptive ranking of multi-modal documents. (December 2015)

• Record Type: Journal Article
• Authors: Akbari, Mohammad; Nie, Liqiang; Chua, Tat-Seng
• Is Part Of: International journal of multimedia information retrieval. Volume 4:Number 4(2015)
• Page Start: 233
• Page End: 245
• Publication Date: 2015-12
• Subjects: Information reranking -- Adaptive ranking -- Multi-modal multi-view search; Information retrieval -- Periodicals; Multimedia systems -- Periodicals; 025.524
• DOI: 10.1007/s13735-015-0088-x
• Languages: English
• ISSNs: 2192-6611

Abstract

Abstract Information reranking aims to recover the true order of the initial search results. Traditional reranking approaches have achieved great success in uni-modal document retrieval. They, however, suffer from the following limitations when reranking multi-modal documents: (1) they are unable to capture and model the relations among multiple modalities within the same document; (2) they usually concatenate diverse features extracted from different modalities into one single vector, rather than adaptively fusing them by considering their discriminative capabilities with respect to the given query; and (3) most of them consider the pairwise relations among documents but discard their higher-order grouping relations, which leads to information loss. Towards this end, we propose an adaptive multi-modal multi-view ( $$\mathbf{aMM }$$ aMM ) reranking model. This model is able to jointly regularize the relatedness among modalities, the effects of feature views extracted from different modalities, as well as the complex relations among multi-modal documents. Extensive experiments on three datasets well validated the effectiveness and robustness of our proposed model.