Tripool: Graph triplet pooling for 3D skeleton-based action recognition. (July 2021)

• Record Type: Journal Article
• Authors: Peng, Wei; Hong, Xiaopeng; Zhao, Guoying
• Is Part Of: Pattern recognition. Volume 115(2021)
• Publication Date: 2021-07
• Subjects: 3D skeletal action recognition -- ST-GCN -- Graph pooling -- Graph topology analysis; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107921
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Research highlights 1: We propose a novel graph pooling method, and for the first time, introduce it into GCNs for skeleton-based action recognition. This method not only reduces the time complexity, as well increases the reception filed. Moreover, it breaks the structure constrain for the high-level features. Research highlights 2: The pooling method provides to optimize a graph triplet loss, in which both graph topology and graph context are captured by our pooling method. We also accelerate this process via minimizing its upper bound. Research highlights 3: We conduct extensive experiments to evaluate this method. Our network constructed with Tripool gets the state-of-the-art performance for the skeleton-based action recognition tasks on two current largest datasets. Abstract: Graph Convolutional Network (GCN) has already been successfully applied to skeleton-based action recognition. However, current GCNs in this task are lack of pooling operations such that the architectures are inherently flat, which not only increases the computational complexity but also requires larger memory space to keep the entire graph embedding. More seriously, a flat architecture forces the high-level semantic feature representations to have the same physical structure of the low-level input skeletons, which we argue is unreasonable and harmful for the final performance. To address these issues, we propose Tripool, a novel graph pooling method for 3D action recognition from skeleton data. Tripool provides to optimize a triplet pooling loss, in which both graph topology and global graph context are taken into consideration, to learn a hierarchical graph representation. The training process of graph pooling is efficient since it optimizes the graph topology by minimizing an upper bound of the pooling loss. Besides, Tripool also automatically generates an embedding matrix since the graph is changed after pooling. On one hand, Tripool reduces the computational cost by removing the redundant nodes. On the other hand it overcomes the limitation of the topology constrain for the high-level semantic representations, thus improves the final performance. Tripool can be combined with various graph neural networks in an end-to-end fashion. Comprehensive experiments on two current largest scale 3D datasets are conducted to evaluate our method. With our Tripool, we consistently get the best results in terms of various performance measures.