A sparse graph wavelet convolution neural network for video-based person re-identification. (September 2022)

• Record Type: Journal Article
• Authors: Yao, Yingmao; Jiang, Xiaoyan; Fujita, Hamido; Fang, Zhijun
• Is Part Of: Pattern recognition. Volume 129(2022)
• Publication Date: 2022-09
• Subjects: Video-based person re-identification -- Weighted sparse graph -- Graph wavelet convolution neural network; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108708
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose an adaptive graph generation module to capture the semantic relationship between different patches across frames to solve the short time occlusion and pedestrian misalignment problems in videobased person Re-ID. The generated sparse weighted graph only captures the supplementary information between highly correlated patches, avoiding the use of pairwise feature mapping that produces information redundancy. We propose a sparse graph wavelet convolution neural network (SGWCNN) framework to model and propagate spatial-temporal relationships between different patches across frames to generate more robust and discriminative features. Compared with traditional GCN-based methods, SGWCNN has sparse parameters, efficient algorithm, and higher accuracy over large-scale video-based person Re-ID datasets. Abstract: Video-based person re-identification (Re-ID) aims to match identical person sequences captured across non-overlapping surveillance areas. It is an essential yet challenging task to effectively embed spatial and temporal information into the video feature representation. For one thing, we observe that different frames in the video can provide complementary information for each other. Also, local features which is lost due to target occlusion or visual ambiguity in one frame can be supplemented by the same pedestrian part in other frames. For another thing, graph neural network enables the contextual interactions between relevant regional features. Therefore, we propose a novel sparse graph wavelet convolution neural network (SGWCNN) for video-based person Re-ID. Distinct from previous graph-based Re-ID methods, we exploit the weighted sparse graph to model the semantic relation among the local patches of pedestrians in the video. Each local patch in one frame can extract supplementary information from highly related patches in other frames. Moreover, to effectively solve the problems of short time occlusion and pedestrian misalignment, the graph wavelet convolution neural network is adopted for feature propagation to refine regional features iteratively. Experiments and evaluation on three challenging benchmarks, that is, MARS, DukeMTMC-VideoReID, and iLIDS-VID, show that the proposed SGWCNN effectively improves the performance of video-based person re-identification.