A monitoring method of freezing of gait based on multimodal fusion. (April 2023)

• Record Type: Journal Article
• Authors: Li, Bochen; Li, Yan; Sun, Yining; Yang, Xianjun; Zhou, Xu; Yao, Zhiming
• Is Part Of: Biomedical signal processing and control. Volume 82(2023)
• Publication Date: 2023-04
• Subjects: Inertial measurement unit -- Force-sensitive insole -- Monitoring of FOG -- Multimodal fusion -- Deep learning; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2023.104589
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: The gait signals from 32 FOG patients are collected by the IMU and FSI simultaneously. Using DNN to extract complementary features from IMU and FSI signals, respectively. Two different feature fusion schemes are designed and compared. A multi-modal fusion classifier for FOG monitoring is proposed with an F1-score of 0.943. A freezing index ratio (FIR) is proposed for automatic labeling of preFOG categories. Abstract: Freezing of Gait (FOG) is an episodic lower extremity movement disorder that is highly susceptible to falls and carries a serious risk of disability. Monitoring of FOG can assist in the diagnosis and treatment of FOG. Providing appropriate gait guidance along with monitoring can help reduce the frequency and duration of freezing episodes. This study aims to improve the robustness of the monitoring model using multimodal fusion methods. The gait signals from 32 FOG patients are collected by the inertial measurement unit (IMU) and force-sensitive insole (FSI) simultaneously. A multimodal fused FOG monitoring model was constructed by using deep neural networks to extract complementary features from IMU and FSI signals respectively, and feature-level fusion of the two modalities by an adaptive weighting method. Experimental results show that the proposed multimodal fusion approach improves the F1 value by 0.029 in the FOG detection task compared to the unimodal model. In addition, to construct the pre-FOG dataset more accurately, an automatic labeling method of pre-FOG events based on the FOG index ratio is also proposed in this paper. Compared to directly labeling the data 2.5 s before the freezing episode as the pre-FOG event, the proposed labeling method obtained more samples and improved the freezing prediction accuracy by 1.4 %.