Action recognition via pose-based graph convolutional networks with intermediate dense supervision. (January 2022)

• Record Type: Journal Article
• Authors: Shi, Lei; Zhang, Yifan; Cheng, Jian; Lu, Hanqing
• Is Part Of: Pattern recognition. Volume 121(2022)
• Publication Date: 2022-01
• Subjects: Action recognition -- Skeleton; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108170
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a pose-based graph convolutional network (PGCN), which employs the graph convolutional module to model the spatiotemporal correlations among the pose-related features to produce a highly discriminative representation for human action recognition. We point out the laziness problem of the backbone CNN, and further propose a novel intermediate dense supervision (IDS) to solve this problem. It is simple and effective, without the need for extra parameters and computations. We evaluate our approach on three popular benchmarks for pose-based action recognition, where our approach achieves stateof-the-art performance on all of them. Abstract: Pose-based action recognition has drawn considerable attention recently. Existing methods exploit the joint position to extract body-part features from the activation maps of the backbone CNN to assist human action recognition. However, there are two limitations: (1) the body-part features are independently used or simply concatenated to obtain a representation, where the prior knowledge about the structured correlations between body parts are not fully exploited; (2) the backbone CNN, from which the body-part features are extracted, is "lazy". It always contents itself with identifying patterns from the most discriminative areas of the input, which causes no information on the features extracted from other areas. This consequently hampers the performance of the followed aggregation process and makes the model easy to be misled by the training data bias. To address these problems, we encode the body-part features into a human-based spatiotemporal graph and employ a light-weight graph convolutional module to explicitly model the dependencies between body parts. Besides, we introduce a novel intermediate dense supervision to promote the backbone CNN to treat all regions equally, which is simple and effective, without extra parameters and computations. The proposed approach, namely, the pose-based graph convolutional network (PGCN), is evaluated on three popular benchmarks, where our approach significantly outperforms the state-of-the-art methods.