Mitigating Illusory Results through Preregistration in Education. Issue 2 (3rd April 2018)

• Record Type: Journal Article
• Authors: Gehlbach, Hunter; Robinson, Carly D.
• Is Part Of: Journal of research on educational effectiveness. Volume 11:Issue 2(2018)
• Page Start: 296
• Page End: 315
• Publication Date: 2018-04-03
• Subjects: educational research -- p-hacking -- preregistration -- replication -- research methods; Education -- Research -- Periodicals; Education -- Evaluation -- Periodicals; Education -- Methodology -- Periodicals; 371.422
• DOI: 10.1080/19345747.2017.1387950
• Languages: English
• ISSNs: 1934-5747

Abstract

ABSTRACT: Like performance-enhancing drugs inflating apparent athletic achievements, several common social science practices contribute to the production of illusory results. In this article, we examine the processes that lead to illusory findings and describe their consequences. We borrow from an approach used increasingly by other disciplines—the norm of preregistering studies. Specifically, we examine how this practice of publicly posting documentation of one's prespecified hypotheses and other key decisions of a study prior to study implementation or data analysis could improve scientific integrity within education. In an attempt to develop initial guidelines to facilitate preregistrations in education, we discuss the types of studies that ought to be preregistered and the logistics of how educational researchers might execute preregistrations. We conclude with ideas for how researchers, reviewers, and the field of education more broadly might speed the adoption of this new norm.