A multilevel study of position effects in PISA achievement tests: student- and school-level predictors in the German tracked school system. Issue 4 (4th July 2019)

• Record Type: Journal Article
• Authors: Nagy, Gabriel; Nagengast, Benjamin; Frey, Andreas; Becker, Michael; Rose, Norman
• Is Part Of: Assessment in education. Volume 26:Issue 4(2019)
• Page Start: 422
• Page End: 443
• Publication Date: 2019-07-04
• Subjects: PISA 2006 -- position effect -- predictors of position effects -- multilevel modelling -- validity; Educational tests and measurements -- Periodicals; Education -- Evaluation -- Periodicals; 371.26
• DOI: 10.1080/0969594X.2018.1449100
• Languages: English
• ISSNs: 0969-594X

Abstract

Abstract: Position effects (PE) cause decreasing probabilities of correct item responses towards the end of a test. We analysed PEs in science, mathematics and reading tests administered in the German extension to the PISA 2006 study with respect to their variability at the student- and school-level. PEs were strongest in reading and weakest in mathematics. Variability in PEs was found at both levels of analysis. PEs were stronger for male students, for students with a migration background (science and mathematics), and for students with a less favourable socio-economic background (reading). At the school level, PEs were stronger in lower school tracks and in schools with a high proportion of students with a migration background. The relationships of the test scores with the covariates partly reflected the covariates' relationships with PEs. Our findings suggest that PEs should be taken seriously in large-scale assessments as they have an undesirable impact on the results.