Gesture recognition using dual-stream CNN based on fusion of sEMG energy kernel phase portrait and IMU amplitude image. (March 2022)

• Record Type: Journal Article
• Authors: Xu, Liukai; Zhang, Keqin; Yang, Genke; Chu, Jian
• Is Part Of: Biomedical signal processing and control. Volume 73(2022)
• Publication Date: 2022-03
• Subjects: sEMG -- Gesture recognition -- Energy kernel -- CNN -- IMU; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2021.103364
• Languages: English
• ISSNs: 1746-8094

Abstract

Graphical abstract: Highlights: Energy kernel phase portrait is extracted for sEMG image processing. Amplitude image method is used for IMU data processing instead of calculating the angles. The dual-stream CNN is built in the feature fusion of sEMG and IMU. Abstract: Since the surface electromyography signal (sEMG) is a weak and unstable signal, human action recognition based on sEMG time domain or frequency domain feature extraction has poor stability and low recognition rate. Based on modeling analysis of sEMG, the dual-stream convolutional neural network (CNN) is built to perform feature fusion of sEMG energy kernel phase portrait and inertial measurement unit(IMU) data for gesture recognition in this paper. Firstly, methods of matrix counting method and time window amplitude are utilized to represent the sEMG and IMU data as images. Secondly, the moving average method is presented to filter the images. Thirdly, the dual-stream CNN is used to perform feature extraction, fusion and classification of sEMG and IMU images respectively. Finally, the experiments yield an average recognition accuracy of 95.78% for 6 gestures performed by 5 subjects, which demonstrates that the proposed method has better performance for human gestures recognition.