A physiology-based approach for estimation of mental fatigue levels with both high time resolution and high level of granularity. (2021)

• Record Type: Journal Article
• Authors: Hu, Xianyin; Nakatsuru, Shinji; Ban, Yuki; Fukui, Rui; Warisawa, Shin'ichi
• Is Part Of: Informatics in medicine unlocked. Volume 24(2021)
• Publication Date: 2021
• Subjects: Mental fatigue -- Long-short Term Memory -- Physiological signals -- High granularity level -- Real-time estimation -- Accumulated local effects; Medical informatics -- Periodicals; 610.285
• DOI: 10.1016/j.imu.2021.100594
• Languages: English
• ISSNs: 2352-9148

Abstract

Abstract: Mental fatigue (MF) monitoring is essential for eliminating accidents in high-risk tasks and providing better productivity management in daily work tasks involving human operators. Previous works only built MF monitoring systems with either high time resolution or high granularity level. We proposed a physiological-based approach to estimate MF Level every 2 seconds in a regression manner, a system that realized both high time resolution and high granularity level. The approach consists of an accurate MF level assessment method using the alignment score within a modified N-back task that is free from the lure effect, a long short-term memory (LSTM) deep learning framework, and a performance-reliability validation process. We used multiple physiological signals including ECG, respiration, and pupil diameter. As a result, we provided feasible estimating performance not worse than existing studies while realizing both high time resolution and high level of granularity. The interpretation analysis utilized accumulated local effects (ALE) to illustrate how black-box models make estimations and to improve the reliability of this approach. Highlights: A mental fatigue (MF) monitoring system with both high time resolution and high granularity level achieved. Developed a MF level measurement method within a modified N-back task with the lure effect eliminated using alignment score. High estimation performance of RMSE = 0.13 and R = 0.7 obtained with unexplored deep learning LSTM model. Interpretation analysis using accumulated local effects provided reliability of the black-box model.