Biosignal-based transferable attention Bi-ConvGRU deep network for hand-gesture recognition towards online upper-limb prosthesis control. (September 2022)

• Record Type: Journal Article
• Authors: Xie, Baao; Meng, James; Li, Baihua; Harland, Andy
• Is Part Of: Computer methods and programs in biomedicine. Volume 224(2022)
• Publication Date: 2022-09
• Subjects: Upper-limb prosthesis control -- Bio-signal analysis -- Biomedicine and informatics -- Deep learning -- Gesture recognition -- Transfer learning; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2022.106999
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: A hybrid bidirectional ConvGRU model was developed to extract muscle activation correlation both forwards and backwards in time and across multiple channels. A novel attention mechanism was embedded to extract the intra-attention weights of the signal, and thus increase model robustness to muscle crosstalk and activation diversity of individual subjects. Window based time-domain statistical features were employed to reduce computation cost and increase model robustness to inevitable data noise. The attention-based Bi-ConvGRU model achieves best performance compared with state-of-art models on same datasets A method for transfer learning was proposed and evaluated. It demonstrated that a model pre-trained from non-amputees could be effectively adapted to amputees, therefore reducing data requirements to train a new model or build a personalised model in practice. Abstract: Background and Objective: Upper-limb amputation can significantly affect a person's capabilities with a dramatic impact on their quality of life. As a biological signal, surface electromyogram (sEMG) provides a non-invasive means to measure underlying muscle activation patterns, corresponding to specific hand gestures. This project aims to develop a real-time deep learning based recognition model to automatically and reliably recognise these complex signals of a wide range of daily hand gestures from amputees and non-amputees. Methods: This paper proposes an attention bidirectional Convolutional Gated Recurrent Unit (Bi-ConvGRU) deep neural network for hand-gesture recognition. By training on sEMG data from both amputees and non-amputees, the model can learn to recognise a group of fine-grained hand movements. This is a significantly more challenging and underexplored area, compared to existing studies on coarse-control in lower limbs. One dimensional CNNs are initially used to extract intra-channel features. The novel use of a bidirectional sequential GRU (Bi-GRU) deep neural network allows the exploration of correlation of muscle activation among multi-channel sEMG signals from both prior and posterior time sequences. Importantly, the attention mechanism is employed following Bi-GRU layers. This enables the model to learn vital parts and feature weights, increasing robustness to bio-data noise and irregularity. Finally, we introduce the first of its kind transfer learning, demonstrating that a baseline model pre-trained with non-amputee data can be effectively refined with amputee data to build a personalised model for amputees. Results: The attention Bi-ConvGRU was evaluated on the benchmark database Ninapro, and achieved an average accuracy of 88.7%, outperforming the state-of-the-art on 18 gesture recognition by 6.7%. Conclusions: To our knowledge, the developed end-to-end deep learning model is the first of its kind that enables reliable predictive decision making in short time windows (160ms). This reduced latency limits physiological awareness, enabling the potential for real-time, online and thus more intuitive bio-control of prosthetic devices for amputees.