Cross-modality person re-identification via multi-task learning. (August 2022)

• Record Type: Journal Article
• Authors: Huang, Nianchang; Liu, Kunlong; Liu, Yang; Zhang, Qiang; Han, Jungong
• Is Part Of: Pattern recognition. Volume 128(2022)
• Publication Date: 2022-08
• Subjects: Cross-modality person re-identification -- Person body information -- Multi-task learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108653
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We take the initiative to investigate the importance and strategy of exploiting person body information for cross-modality VI-PReID. We formulate a multi-task learning model by building each individual branch for a different recognition task relevant to person Re-ID. We design a task translation module to transfer the person body information from the person mask prediction branch into the VI-PReID branch. Abstract: Despite its promising preliminary results, existing cross-modality Visible-Infrared Person Re-IDentification (VI-PReID) models incorporating semantic (person) masks simply use these person masks as selection maps to separate person features from background regions. Such models do not dedicate to extracting more modality-invariant person body features in the VI-PReID network itself, thus leading to suboptimal results in VI-PReID. Differently, we aim to better capture person body information in the VI-PReID network itself for VI-PReID by exploiting the inner relations between person mask prediction and VI-PReID. To this end, a novel multi-task learning model is presented in this paper, where person body features obtained by person mask prediction potentially facilitate the extraction of discriminative modality-shared person body information for VI-PReID. On top of that, considering the task difference between person mask prediction and VI-PReID, we propose a novel task translation sub-network to transfer discriminative person body information, extracted by person mask prediction, into VI-PReID. Doing so enables our model to better exploit discriminative and modality-invariant person body information. Thanks to more discriminative modality-shared features, our method outperforms previous state-of-the-arts by a significant margin on several benchmark datasets. Our intriguing findings validate the effectiveness of extracting discriminative person body features for the VI-PReID task.