Integrating the Analysis of Mental Operations Into Multilevel Models to Validate an Assessment of Higher Education Students' Competency in Business and Economics. Issue 3 (August 2016)

• Record Type: Journal Article
• Authors: Brückner, Sebastian; Pellegrino, James W.
• Other Names: Pant Hans Anand guestEditor.; Zlatkin‐Troitschanskaia Olga guestEditor.
• Is Part Of: Journal of educational measurement. Volume 53:Issue 3(2016)
• Page Start: 293
• Page End: 312
• Publication Date: 2016-08
• Subjects: Educational tests and measurements -- Periodicals; 371.2605
• DOI: 10.1111/jedm.12113
• Languages: English
• ISSNs: 0022-0655

Abstract

Abstract : The Standards for Educational and Psychological Testing indicate that validation of assessments should include analyses of participants' response processes. However, such analyses typically are conducted only to supplement quantitative field studies with qualitative data, and seldom are such data connected to quantitative data on student or item performance. This paper presents an example of how data from an analysis of mental operations collected using a sociocognitive approach can be quantitatively integrated with other data on student and item performance to validate in part an assessment of higher education students' competency in business and economics. Evidence of forward reasoning and paraphrasing as mental operations is obtained using the think‐aloud method. As part of the validity argument and to enhance credibility of the findings, the generalized linear models are expressed as multilevel models in which the analyses of response processes are aligned with quantitative findings from large‐scale field studies.