Multi-sensory assessment for hand pattern recognition. (February 2022)

• Record Type: Journal Article
• Authors: Amrani, Mohamed Z.; Borst, Christoph W.; Achour, Nouara
• Is Part Of: Biomedical signal processing and control. Volume 72(2022)Part B
• Publication Date: 2022-02
• Subjects: Classification -- EMG signals -- Hand pattern recognition -- Multi-sensory; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2021.103368
• Languages: English
• ISSNs: 1746-8094

Abstract

Abstract: This study presents and assesses a multi-sensory hand pattern recognizer. We experimentally evaluated multiple features from the selected sensors, detected the most significant ones, and set the best combination possible to reach optimal hand shape detection. multilayer perceptron (MLP) and Decision Tree (DT) classifiers were used, and we used these two classification techniques to ensure that the results were not related to the classification technique. Leap Motion (LM) controllers and a VIVE Pro Eye camera were used to detect hand keypoints, and those last sensors are vision-based. We integrated this with a MYO armband, which streams eight EMG channels and inertial measurement unit data. All the sensors' data were synchronized and streamed in real-time to be treated and fused. We evaluated our integration on the classification of the American Sign Language (ASL). We worked with seven participants, and we detected the best features subset. The overall results were satisfactory, wherein we reached an average classification of 98.31% with the optimal subset.