Gesture recognition from RGB images using convolutional neural network‐attention based system. (26th July 2022)

• Record Type: Journal Article
• Authors: Barbhuiya, Abul Abbas; Karsh, Ram Kumar; Jain, Rahul
• Is Part Of: Concurrency and computation. Volume 34:Number 24(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-07-26
• Subjects: CNN -- deep learning -- hand gesture -- human computer interaction -- sign language recognition system; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.7230
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: Hand gestures can be categorized based on their applications as conversational, manipulative, controlling, and communicative gestures. Hand gesture recognition is an aspect of human action recognition, which plays a notable role in communication among the deaf and dumb community. Accurate recognition and classification of hand gestures with similar postures is still a complex problem. The main objective of this work is to employ deep learning based convolutional neural networks for human action recognition. This article introduces a convolutional neural network based on VGG16 architecture with an attention approach to mitigate the issue. Incorporating an attention‐based module with the VGG16 architecture is the prime reason for potentially learning distinguishing image features by the network. Besides, the proposed robust system collectively recognizes and classifies all the available 36 hand gesture classes of the Massey database, where most previous work selected only a few distinguishable gestures. Experimental results show that the average recognition accuracy for characters with similar hand postures like "m" and "n" is better than the state‐of‐the‐art networks. Additionally, the proposed method attains about 3% higher recognition accuracy than the state‐of‐the‐art networks for all gesture classes. The efficacy of the proposed architecture has also been validated via precision, recall, and F‐score.