Machine learning-based analysis of operator pupillary response to assess cognitive workload in clinical ultrasound imaging. (August 2021)

• Record Type: Journal Article
• Authors: Sharma, Harshita; Drukker, Lior; Papageorghiou, Aris T.; Noble, J. Alison
• Is Part Of: Computers in biology and medicine. Volume 135(2021)
• Publication Date: 2021-08
• Subjects: Eye-tracking -- Pupillometry -- Cognitive workload -- Ultrasound imaging -- Time-series analysis -- Sonography data science -- Fetal ultrasound -- Machine learning -- Deep learning -- Convolutional neural network -- Multi-modal data; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2021.104589
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Introduction: Pupillometry, the measurement of eye pupil diameter, is a well-established and objective modality correlated with cognitive workload. In this paper, we analyse the pupillary response of ultrasound imaging operators to assess their cognitive workload, captured while they undertake routine fetal ultrasound examinations. Our experiments and analysis are performed on real-world datasets obtained using remote eye-tracking under natural clinical environmental conditions. Methods: Our analysis pipeline involves careful temporal sequence (time-series) extraction by retrospectively matching the pupil diameter data with tasks captured in the corresponding ultrasound scan video in a multi-modal data acquisition setup. This is followed by the pupil diameter pre-processing and the calculation of pupillary response sequences. Exploratory statistical analysis of the operator pupillary responses and comparisons of the distributions between ultrasonographic tasks (fetal heart versus fetal brain) and operator expertise (newly-qualified versus experienced operators) are performed. Machine learning is explored to automatically classify the temporal sequences into the corresponding ultrasonographic tasks and operator experience using temporal, spectral, and time-frequency features with classical (shallow) models, and convolutional neural networks as deep learning models. Results: Preliminary statistical analysis of the extracted pupillary response shows a significant variation for different ultrasonographic tasks and operator expertise, suggesting different extents of cognitive workload in each case, as measured by pupillometry. The best-performing machine learning models achieve receiver operating characteristic (ROC) area under curve (AUC) values of 0.98 and 0.80, for ultrasonographic task classification and operator experience classification, respectively. Conclusion: We conclude that we can successfully assess cognitive workload from pupil diameter changes measured while ultrasound operators perform routine scans. The machine learning allows the discrimination of the undertaken ultrasonographic tasks and scanning expertise using the pupillary response sequences as an index of the operators' cognitive workload. A high cognitive workload can reduce operator efficiency and constrain their decision-making, hence, the ability to objectively assess cognitive workload is a first step towards understanding these effects on operator performance in biomedical applications such as medical imaging. Highlights: Machine learning-based pupillary response analysis is performed to assess operator cognitive workload in clinical ultrasound. A systematic multi-modal data analysis pipeline is proposed using eye-tracking, pupillometry, and sonography data science. Pertinent challenges of natural or real-world clinical datasets are addressed. Pupillary responses around event triggers, different ultrasonographic tasks, and different operator experiences are studied. Machine learning models are learnt to classify undertaken tasks or operator expertise from pupillometric time-series data.