ViT-LLMR: Vision Transformer-based lower limb motion recognition from fusion signals of MMG and IMU. (April 2023)

• Record Type: Journal Article
• Authors: Zhang, Hanyang; Yang, Ke; Cao, Gangsheng; Xia, Chunming
• Is Part Of: Biomedical signal processing and control. Volume 82(2023)
• Publication Date: 2023-04
• Subjects: Mechanomyography -- Vision Transformer -- Attention mechanism -- Signal fusion; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.104508
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: A Vision Transformer-based architecture for lower limb motion recognition from MMG and kinematic signals was proposed. The model can recognize 8 motions with an accuracy of 94.62%, much higher than machine learning methods and CNN. Fusion signals of MMG and kinematic data improve the classification accuracies of the model. The proposed ViT-LLMR maintains high recognition accuracy when undersampling and only use part of signals. Abstract: One of the key problems in lower limb-based human–computer interaction (HCI) technology is to use wearable devices to recognize the wearer's lower limb motions. The information commonly used to discriminate human motion mainly includes biological and kinematic signals. Considering that unimodal signals do not provide enough information to recognize lower limb movements, in this paper, we proposed a Vision Transformer (ViT)-based architecture for lower limb motion recognition from multichannel Mechanomyography (MMG) signals and kinematic data. Firstly, we applied the self-attention mechanism to enhance each input channel signal. Then the data was fed into ViT model. Vision Transformer-based Lower Limb Motion Recognition (ViT - LLMR) architecture proposed in this paper can avoid the model training problems such as autonomous feature extraction and feature selection for machine learning, and the model can recognize eight lower limb motions containing six subjects with an accuracy of 94.62%. In addition, we analyzed the generalization ability of the model when undersampling and only collecting fragment signals. In conclusion, the proposed ViT - LLMR architecture could provide a basis for practical applications in different HCI fields.