Sequential inter-hop graph convolution neural network (SIhGCN) for skeleton-based human action recognition. (1st June 2022)

• Record Type: Journal Article
• Authors: Setiawan, Feri; Yahya, Bernardo Nugroho; Chun, Seok-Ju; Lee, Seok-Lyong
• Is Part Of: Expert systems with applications. Volume 195(2022)
• Publication Date: 2022-06-01
• Subjects: Action recognition -- Attention mechanism -- Feature aggregation -- Graph convolutional neural network -- Normalized Laplacian matrix; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.116566
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: A graph convolution model for skeleton-based action recognition is proposed. Normalized Laplacian Matrix is utilized to encode the graph information. An attention-based feature aggregation is proposed to extract the salient features. The proposed method achieves better results than the baseline models. Abstract: Skeleton-based human action recognition has attracted a lot of attention due to its capability and potential to provide more information than just using the sequence of RGB images. The use of Graph Convolutional Neural Network (GCN) becomes more popular since it can model the human skeleton very well. However, the existing GCN architectures ignore the different levels of importance on each hop during the feature aggregation and use the final hop information for further calculation, resulting in considerable information loss. Besides, they use the standard Laplacian or adjacency matrix to encode the property of a graph into a set of vectors which has a limitation in terms of graph invariants. In this work, we propose a Sequential Inter-hop Graph Convolution Neural Network (SIhGCN) which can capture salient graph information from every single hop rather than the final hop only and our work utilizes the normalized Laplacian matrix which provides better representation since it relates well to graph invariants. The proposed method is validated on two large datasets, NTU-RBG + D and Kinetics, to demonstrate the superiority of our proposed method.