Applying the total survey error framework to PIAAC. Issue 2 (3rd April 2018)

• Record Type: Journal Article
• Authors: Lyberg, Lars; Cibelli Hibben, Kristen; Pennell, Beth-Ellen
• Is Part Of: Quality assurance in education. Volume 26:Issue 2(2018)
• Page Start: 153
• Page End: 168
• Publication Date: 2018-04-03
• Subjects: Quality control -- Quality assurance -- Cross-national surveys -- Data quality -- Total survey error; Education -- Standards -- Periodicals; 370
• DOI: 10.1108/QAE-07-2017-0035
• Languages: English
• ISSNs: 0968-4883

Abstract

Abstract : Purpose: Surveys in multinational, multiregional and multicultural contexts (or "3MC" surveys) are becoming increasingly important to global and regional decision-making and theory building. To serve this purpose, the surveys need to be well managed, with an awareness of key sources of survey error and how to minimize them, mechanisms in place to control the implementation process and an ability to intervene in that process when necessary in a spirit of continuous improvement (Pennell et al ., 2017 ). One key approach for managing and assessing the quality of 3MC surveys is the total survey error (TSE) framework and associated survey process quality. This paper aims to examine the application of the TSE framework and survey process quality to the Programme for the International Assessment of Adult Competencies (PIAAC). Design/methodology/approach: The authors begin with a background on TSE and discuss recent adaptations of TSE and survey process quality for 3MC surveys. They then presents a TSE framework tailored with examples of potential contributions to error for PIAAC and ways to address those through effective quality assurance (QA) and quality control (QC) approaches. Findings: Overall, the authors find that the design and implementation of the first cycle of PIAAC largely reflect the current best practice for 3MC surveys. However, the authors identify several potential contributions to error that may threaten comparability in PIAAC and ways these could be addressed in the upcoming cycle. Originality/value: With a view toward continuous improvement, the final section draws on the survey process quality approach adapted fromHansen et al .'s study (2016 ) to summarize the recommendations in terms of additional QA elements (inputs and activities) and associated QC elements (measures and reports) for PIAAC's consideration in the next cycle.