Arbitrary-view human action recognition via novel-view action generation. (October 2021) - Gedamu, Kumie; Ji, Yanli; Yang, Yang; Gao, LingLing; Shen, Heng Tao
Highlights: We propose a two-branch novel-view action generation approach for arbitrary-view action recognition. The two-branch generation model generates novel-view action samples, which enlarges the view range of action samples for classifier training. A view-domain generalization module is designed to weaken the difference of action representation in various views for the arbitrary-view action recognition. Extensive experiments and ablation studies are performed on three large-scale benchmarks, UESTC, NTU-60 and NTU-120 datasets. Abstract: Arbitrary-view human action recognition is still a big challenge due to the view changes. A possible solution is to enlarge the view range of action samples in the training set. Therefore, we propose a Two-Branch Novel-View action Generation approach based on auxiliary conditional GAN, which generates a novel-view action sample for arbitrary-view human action recognition. The generated sample enlarge the view range of action samples for training. Furthermore, to narrow the representation of actions in different views, we propose a view-domain generalization model that improves the recognition performance of arbitrary-view human action recognition. Our approach is evaluated on three large-scale RGB+D skeleton datasets including UESTC varying-view RGB+D dataset, NTU RGB+D 60, and NTU RGB+D 120 datasets, with two types of view-invariant evaluations, i.e., the cross-view, and arbitrary-view recognition. The proposed approach achieves outstanding performance in human action recognition.