What happens when econometrics and psychometrics collide? An example using the PISA data. (December 2017)

• Record Type: Journal Article
• Authors: Jerrim, John; Lopez-Agudo, Luis Alejandro; Marcenaro-Gutierrez, Oscar D.; Shure, Nikki
• Is Part Of: Economics of education review. Volume 61(2017:Dec.)
• Page Start: 51
• Page End: 58
• Publication Date: 2017-12
• Subjects: Sample design -- Test design -- PISA -- Weights -- Replicate weights -- Plausible values; I20 -- C18 -- C10 -- C55; Education -- Economic aspects -- Periodicals; 370
• DOI: 10.1016/j.econedurev.2017.09.007
• Languages: English
• ISSNs: 0272-7757

Abstract

Highlights: International assessment data such as PISA are now widely used by economists, but the complex survey and test design of such studies remain poorly understood. We describe in detail key elements of the PISA study, and the implications this has for secondary analyses of these data. Our replication case study, based upon Lavy (2015), illustrates the impact that ignoring these features has upon one particularly interesting set of empirical results. Abstract: International large-scale assessments such as PISA are increasingly being used to benchmark the academic performance of young people across the world. Yet many of the technicalities underpinning these datasets are misunderstood by applied researchers, who sometimes fail to take their complex sample and test designs into account. The aim of this paper is to generate a better understanding among economists about how such databases are created, and what this implies for the empirical methodologies one should (or should not) apply. We explain how some of the modeling strategies preferred by economists seem to be at odds with the complex test design, and provide clear advice on the types of robustness tests that are therefore needed when analyzing these datasets. In doing so, we hope to generate a better understanding of international large-scale education databases, and promote better practice in their use.