Correcting for sample problems in PISA and the improvement in Portuguese students' performance. Issue 4 (1st October 2016)

• Record Type: Journal Article
• Authors: Freitas, Pedro; Nunes, Luís Catela; Balcão Reis, Ana; Seabra, Carmo; Ferro, Adriana
• Is Part Of: Assessment in education. Volume 23:Issue 4(2016)
• Page Start: 456
• Page End: 472
• Publication Date: 2016-10-01
• Subjects: PISA -- samples -- educational achievement -- age-based assessment -- participation rates; I20 -- 125; Educational tests and measurements -- Periodicals; Education -- Evaluation -- Periodicals; 371.26
• DOI: 10.1080/0969594X.2015.1105784
• Languages: English
• ISSNs: 0969-594X

Abstract

Abstract : The results of large-scale international assessments such as Programme for International Student Assessment (PISA) have attracted a considerable attention worldwide and are often used by policy-makers to support educational policies. To ensure that the published results represent the actual population, these surveys go through a thorough scrutiny to ensure their validity. One important issue that may affect the results is the occurrence of different participation rates across groups with heterogeneous average student scores. In this study, we illustrate how problems of representativeness of the samples may be corrected by the use of post-stratified weights. We use the case of Portugal, a country where we uncover a meaningful difference between the actual population and the PISA samples in terms of the distribution of students per grade and track of studies. We show that after post-stratification, the temporal evolution of the PISA scores is quite different from the one reported.