An optimized Generative Adversarial Network based continuous sign language classification. (15th November 2021)

• Record Type: Journal Article
• Authors: Elakkiya, R.; Vijayakumar, Pandi; Kumar, Neeraj
• Is Part Of: Expert systems with applications. Volume 182(2021)
• Publication Date: 2021-11-15
• Subjects: Continuous sign language recognition -- Generative Adversarial Networks -- Sign classification -- Feature dimensionality reduction -- Hyperparameter optimization; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115276
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Characterization of manual and non-manual gestures in recognizing the sign gestures. Deep Networks of self-learning capacity to achieve higher recognition rate. Iterative optimization on hyperparameters and considered limited training data. Recognize multimodal and multilingual sign corpus with multi-signer variation. Abstract: Classifying manual and non-manual gestures in sign language recognition is a complex and challenging task. Sign language gestures are the combination of hand, face, and body postures, which often have self-occlusions and inter-object occlusions of both the hands, hands with face, or hands with upper body postures. This paper addresses the characterization of manual and non-manual gestures in recognizing the sign language gestures from continuous video sequences. This paper introduces a novel hyperparameter based optimized Generative Adversarial Networks (H-GANs) to classify the sign gestures, and it works in three phases. In phase-I, it adapts the stacked variational auto-encoders (SVAE) and Principal Component Analysis (PCA) to get the pre-tuned data with reduced feature dimensions. In Phase-II, the H-GANs employed Deep Long Short Term Memory (LSTM) as generator and LSTM with 3D Convolutional Neural Network (3D-CNN) as a discriminator. The generator generates random sequences with noise from the real sequence of frames, and the discriminator detects and classifies the real frames of sign gestures. In Phase-III, the proposed approach employs Deep Reinforcement Learning (DRL) for hyperparameter optimization and regularization. By getting the reward points, Proximal Policy Optimization (PPO) optimizes the hyperparameters, and Bayesian Optimization (BO) regularizes the hyperparameters. The proposed H-GANs used two different large vocabulary sign corpus of continuous sign videos to evaluate the performance and efficiency of the system. The experimental results on different dimensions reveal that the H-GANs improved the accuracy and recognition rate when compared with the state-of-the-art classification methods with reduced complexity.