Protecting the integrity of survey research. Issue 3 (28th March 2023)

• Record Type: Journal Article
• Authors: Jamieson, Kathleen Hall; Lupia, Arthur; Amaya, Ashley; Brady, Henry E; Bautista, René; Clinton, Joshua D; Dever, Jill A; Dutwin, David; Goroff, Daniel L; Hillygus, D Sunshine; Kennedy, Courtney; Langer, Gary; Lapinski, John S; Link, Michael; Philpot, Tasha; Prewitt, Ken; Rivers, Doug; Vavreck, Lynn; Wilson, David C; McNutt, Marcia K
• Editors: Gelfand, Michele
• Is Part Of: PNAS nexus. Volume 2:Issue 3(2023)
• Publication Date: 2023-03-28
• Subjects: scientific integrity -- survey research -- weighting of data; Science -- Periodicals; 505
• DOI: 10.1093/pnasnexus/pgad049
• Languages: English
• ISSNs: 2752-6542

Abstract

Abstract: Although polling is not irredeemably broken, changes in technology and society create challenges that, if not addressed well, can threaten the quality of election polls and other important surveys on topics such as the economy. This essay describes some of these challenges and recommends remediations to protect the integrity of all kinds of survey research, including election polls. These 12 recommendations specify ways that survey researchers, and those who use polls and other public-oriented surveys, can increase the accuracy and trustworthiness of their data and analyses. Many of these recommendations align practice with the scientific norms of transparency, clarity, and self-correction. The transparency recommendations focus on improving disclosure of factors that affect the nature and quality of survey data. The clarity recommendations call for more precise use of terms such as "representative sample" and clear description of survey attributes that can affect accuracy. The recommendation about correcting the record urges the creation of a publicly available, professionally curated archive of identified technical problems and their remedies. The paper also calls for development of better benchmarks and for additional research on the effects of panel conditioning. Finally, the authors suggest ways to help people who want to use or learn from survey research understand the strengths and limitations of surveys and distinguish legitimate and problematic uses of these methods.