A hierarchical latent response model for inferences about examinee engagement in terms of guessing and item‐level non‐response. (10th November 2019)

• Record Type: Journal Article
• Authors: Ulitzsch, Esther; von Davier, Matthias; Pohl, Steffi
• Is Part Of: British journal of mathematical & statistical psychology. Volume 73(2020)Supplement 1
• Page Start: 83
• Page End: 112
• Publication Date: 2019-11-10
• Subjects: response times -- missing responses -- item response theory -- engagement -- guessing; Psychometrics -- Periodicals; Psychology -- Mathematical models -- Periodicals; Psychology -- Statistical methods -- Periodicals; 150.727
• DOI: 10.1111/bmsp.12188
• Languages: English
• ISSNs: 0007-1102

Abstract

Abstract : In low‐stakes assessments, test performance has few or no consequences for examinees themselves, so that examinees may not be fully engaged when answering the items. Instead of engaging in solution behaviour, disengaged examinees might randomly guess or generate no response at all. When ignored, examinee disengagement poses a severe threat to the validity of results obtained from low‐stakes assessments. Statistical modelling approaches in educational measurement have been proposed that account for non‐response or for guessing, but do not consider both types of disengaged behaviour simultaneously. We bring together research on modelling examinee engagement and research on missing values and present a hierarchical latent response model for identifying and modelling the processes associated with examinee disengagement jointly with the processes associated with engaged responses. To that end, we employ a mixture model that identifies disengagement at the item‐by‐examinee level by assuming different data‐generating processes underlying item responses and omissions, respectively, as well as response times associated with engaged and disengaged behaviour. By modelling examinee engagement with a latent response framework, the model allows assessing how examinee engagement relates to ability and speed as well as to identify items that are likely to evoke disengaged test‐taking behaviour. An illustration of the model by means of an application to real data is presented.