What-and-where to match: Deep spatially multiplicative integration networks for person re-identification. (April 2018)

• Record Type: Journal Article
• Authors: Wu, Lin; Wang, Yang; Li, Xue; Gao, Junbin
• Is Part Of: Pattern recognition. Volume 76(2018:Apr.)
• Page Start: 727
• Page End: 738
• Publication Date: 2018-04
• Subjects: Multiplicative integration gating -- Convolutional neural networks -- Recurrent neural networks -- Person re-identification; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.10.004
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel deep architecture to emphasize common local patterns is proposed to learn flexible joint representations for person re-identification. The proposed method introduces a multiplicative integration gating function to embed two convolutional features to their joint representations, which are effective in discriminating positive pairs from negative pairs. Spatial dependencies are incorporated into feature learning to address the cross-view misalignment. Extensive experiments and empirical analysis are provided in experimental part. Abstract: Matching pedestrians across disjoint camera views, known as person re-identification (re-id), is a challenging problem that is of importance to visual recognition and surveillance. Most existing methods exploit local regions with spatial manipulation to perform matching in local correspondences. However, they essentially extract fixed representations from pre-divided regions for each image and then perform matching based on these extracted representations. For models in this pipeline, local finer patterns that are crucial to distinguish positive pairs from negative ones cannot be captured, and thus making them underperformed. In this paper, we propose a novel deep multiplicative integration gating function, which answers the question of what-and-where to match for effective person re-id. To address what to match, our deep network emphasizes common local patterns by learning joint representations in a multiplicative way. The network comprises two Convolutional Neural Networks (CNNs) to extract convolutional activations, and generates relevant descriptors for pedestrian matching. This leads to flexible representations for pair-wise images. To address where to match, we combat the spatial misalignment by performing spatially recurrent pooling via a four-directional recurrent neural network to impose spatial dependency over all positions with respect to the entire image. The proposed network is designed to be end-to-end trainable to characterize local pairwise feature interactions in a spatially aligned manner. To demonstrate the superiority of our method, extensive experiments are conducted over three benchmark data sets: VIPeR, CUHK03 and Market-1501.