A Method for Identifying Partial Test-Taking Engagement. Issue 2 (3rd April 2021)

• Record Type: Journal Article
• Authors: Wise, Steven; Kuhfeld, Megan
• Is Part Of: Applied measurement in education. Volume 34:Issue 2(2021)
• Page Start: 150
• Page End: 161
• Publication Date: 2021-04-03
• Subjects: Educational tests and measurements -- United States -- Periodicals; Educational tests and measurements -- Periodicals; Education -- Standards -- Periodicals; 371.26
• DOI: 10.1080/08957347.2021.1890745
• Languages: English
• ISSNs: 0895-7347

Abstract

ABSTRACT: Effort-moderated (E-M) scoring is intended to estimate how well a disengaged test taker would have performed had they been fully engaged. It accomplishes this adjustment by excluding disengaged responses from scoring and estimating performance from the remaining responses. The scoring method, however, assumes that the remaining responses are not affected by disengagement. A recent study provided evidence that the responses used in E-M scoring can sometimes reflect less-than-full, or partial engagement. In these instances, E-M scores will be less trustworthy because they can still be distorted by disengagement. This paper describes a method for identifying partial engagement and provides validation evidence to support its use and interpretation. When test events indicate the presence of partial engagement, effort-moderated scores should be interpreted cautiously.