G2DA: Geometry-guided dual-alignment learning for RGB-infrared person re-identification. (March 2023)

• Record Type: Journal Article
• Authors: Wan, Lin; Sun, Zongyuan; Jing, Qianyan; Chen, Yehansen; Lu, Lijing; Li, Zhihang
• Is Part Of: Pattern recognition. Volume 135(2023)
• Publication Date: 2023-03
• Subjects: Person re-identification -- Cross-modality matching -- optimal transport -- Feature alignment -- Channel exchange; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109150
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Jointly semantic and structure alignment is proposed as a new perspective for cross-modality person-ReID. A unified G2DA framework is built to discover modality correspondence by matching the extracted skeleton graphs of cross-modality image pairs. A novel message fusion Attention mechanism is advanced to adaptively strengthen the discriminative power of learned representations. Extensive experiments show that our approach outperforms state-of-the-art methods. Abstract: RGB-Infrared (IR) person re-identification aims to retrieve person-of-interest from heterogeneous cameras, easily suffering from large image modality discrepancy caused by different sensing wavelength ranges. Existing works usually minimize such discrepancy by aligning modality distribution of global features, while neglecting deep semantics and high-order structural relations within each class. This might render the misalignment between heterogeneous samples. In this paper, we propose Geometry-Guided Dual-Alignment (G 2 DA) learning, which yields better sample-level modality alignment for RGB-IR ReID by solving a graph-enabled distribution matching task that maximizes agreement between multi-modality node representations considering edge topology. Specifically, we covert RGB/IR images into semantic-aligned graphs, in which whole-part features and their similarities are represented by nodes and associated edges, respectively. To simultaneously implement node- and edge-wise alignment (Dual Alignment), we introduce Optimal Transport (OT) as a metric to calculate cross-modality human body matching scores. By minimizing the displacement cost across RGB-IR graphs, G 2 DA could learn not just modality-invariant but structurally consistent cross-modality representations. Furthermore, we advance a Message Fusion Attention (MFA) mechanism to adaptively smooth the node representations within each RGB/IR graph, effectively alleviating occlusions caused by other individuals and/or objects. Extensive experiments on two standard benchmark datasets validate the superiority of G 2 DA, yielding competitive performance against previous state-of-the-arts.