Wandering minds, wandering mice: Computer mouse tracking as a method to detect mind wandering. (November 2020)

• Record Type: Journal Article
• Authors: Dias da Silva, M.R.; Postma, M.
• Is Part Of: Computers in human behavior. Volume 112(2020)
• Publication Date: 2020-11
• Subjects: Mouse tracking -- Mind wandering -- Attention -- Embodied cognition -- Working memory -- Arousal; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2020.106453
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: Mind wandering is a state in which an individual's attention is decoupled from the task at hand. Mind wandering affects performance in many tasks requiring focused attention, including online learning. Previous studies have focused primarily on investigating mind wandering in contexts which are conducive to mind wandering, that is, for highly repetitive and monotonous tasks or during tasks with very low attentional demands. However, mind wandering also occurs during highly engaging and demanding tasks. In this study, we examine whether mouse tracking can be used to predict mind wandering in an engaging task involving classical computer interfaces. Assuming that mouse trajectories towards a particular response on the screen are continuously updated by time-dependent and temporally-dynamic cognitive processes, as a behavioral methodology, mouse tracking can provide unique insights into attentional processes. In our experiment, a total of 272 students completed a mouse-based operation span task, during which their thoughts were probed and their mouse movements recorded. Naive Bayes, Linear Discriminant Analyses, K-Nearest Neighbors, Tree Bag, and Random Forest classifiers were able to predict mind wandering with F1-scores of up to 15% above a random-chance baseline. The results show that hand reach movements can be tracked to detect mind wandering in a user-independent manner in online tasks, thus providing a viable alternative to self-report methods and (neuro)physiological measures. Our finding has relevant implications for a variety of user interfaces which require hand and finger movements for the purposes of human–computer interactions. Highlights: Computer mouse movements were used to predict mind wandering during an online task. When mind wandering, participants took longer to initiate and to complete a response. When mind wandering, participants flipped the cursor more often along the x- and y-axes. More flips indicate that movements were more corrective in nature. Our findings indicate that mind wandering and bodily movements are interdependent.