Arbitrary-view human action recognition via novel-view action generation. (October 2021)

• Record Type: Journal Article
• Authors: Gedamu, Kumie; Ji, Yanli; Yang, Yang; Gao, LingLing; Shen, Heng Tao
• Is Part Of: Pattern recognition. Volume 118(2021)
• Publication Date: 2021-10
• Subjects: Arbitrary-view action recognition -- Novel-view action generation -- View domain generalization; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108043
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a two-branch novel-view action generation approach for arbitrary-view action recognition. The two-branch generation model generates novel-view action samples, which enlarges the view range of action samples for classifier training. A view-domain generalization module is designed to weaken the difference of action representation in various views for the arbitrary-view action recognition. Extensive experiments and ablation studies are performed on three large-scale benchmarks, UESTC, NTU-60 and NTU-120 datasets. Abstract: Arbitrary-view human action recognition is still a big challenge due to the view changes. A possible solution is to enlarge the view range of action samples in the training set. Therefore, we propose a Two-Branch Novel-View action Generation approach based on auxiliary conditional GAN, which generates a novel-view action sample for arbitrary-view human action recognition. The generated sample enlarge the view range of action samples for training. Furthermore, to narrow the representation of actions in different views, we propose a view-domain generalization model that improves the recognition performance of arbitrary-view human action recognition. Our approach is evaluated on three large-scale RGB+D skeleton datasets including UESTC varying-view RGB+D dataset, NTU RGB+D 60, and NTU RGB+D 120 datasets, with two types of view-invariant evaluations, i.e., the cross-view, and arbitrary-view recognition. The proposed approach achieves outstanding performance in human action recognition.