Estimating cognitive load from speech gathered in a complex real-life training exercise. Issue 124 (April 2019)

• Record Type: Journal Article
• Authors: Vukovic, Maria; Sethu, Vidhyasaharan; Parker, Jessica; Cavedon, Lawrence; Lech, Margaret; Thangarajah, John
• Is Part Of: International journal of human-computer studies. Issue 124(2019)
• Page Start: 116
• Page End: 133
• Publication Date: 2019-04
• Subjects: Cognitive load estimation -- Speech features -- Speaker dependence -- Real-world dataset; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2018.12.003
• Languages: English
• ISSNs: 1071-5819

Abstract

Highlights: Exploration of cognitive load estimation techniques on real-world dataset. Cognitive load classifiers built from speech that was self-labelled during tasks. Exploration of speaker dependence for addressing speaker variability within tasks. Demonstration of technique robustness by evaluating benchmarked lab-based dataset. Abstract: Speech-enabled applications are becoming prevalent, providing opportunities for real-time detection of speaker characteristics. Estimation of cognitive load from speech is one type of speaker characteristic that can provide insight into the human state in complex, highly dynamic human-machine teaming scenarios and be used to adapt interaction with the user to their current cognitive state. Cognitive load estimation from speech experiments are typically performed on speech gathered in laboratory settings. By contrast, this research is performed on a real-life dataset that was not created for the purpose of cognitive load assessment. Speech was extracted from recordings of a military simulation exercise in which air battle managers communicated with pilots flying simulated aircraft. This paper assesses whether cognitive load can be estimated from speech self-labelled by exercise participants and collected in a realistic setting, and examines how well cognitive load estimation methods translate from the laboratory setting to the real-world. Analysis suggests that participants' self-assessment of workload at periodic intervals can be used to label speech to create 2-class cognitive load classifiers. The analysis also shows that including some target speaker speech in speaker independent training data results in higher classification accuracy than when classifiers are built solely from speaker dependent data.