Scalable multi-label canonical correlation analysis for cross-modal retrieval. (July 2021)

• Record Type: Journal Article
• Authors: Shu, Xin; Zhao, Guoying
• Is Part Of: Pattern recognition. Volume 115(2021)
• Publication Date: 2021-07
• Subjects: Canonical correlation analysis -- Semantic transformation -- Cross-modal retrieval -- Singular value decomposition; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107905
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We develop a novel framework that can integrate the feature correlation and semantic correlation to boost retrieval performance. A semantic transformation is developed to effectively approximate the label similarity matrix. We propose an efficient learning algorithm which has linear time complexity in terms of the number of samples. Abstract: Multi-label canonical correlation analysis (ml-CCA) has been developed for cross-modal retrieval. However, the computation of ml-CCA involves dense matrices eigendecomposition, which can be computationally expensive. In addition, ml-CCA only takes semantic correlation into account which ignores the cross-modal feature correlation. In this paper, we propose a novel framework to simultaneously integrate the semantic correlation and feature correlation for cross-modal retrieval. By using the semantic transformation, we show that our model can avoid computing the covariance matrix explicitly which is a huge save of computational cost. Further analysis shows that our proposed method can be solved via singular value decomposition which has linear time complexity. Experimental results on three multi-label datasets have demonstrated the accuracy and efficiency of our proposed method.