The process of replication target selection in psychology: what to consider?. Issue 2 (1st February 2023)

• Record Type: Journal Article
• Authors: Pittelkow, Merle-Marie; Field, Sarahanne M.; Isager, Peder M.; van't Veer, Anna E.; Anderson, Thomas; Cole, Scott N.; Dominik, Tomáš; Giner-Sorolla, Roger; Gok, Sebahat; Heyman, Tom; Jekel, Marc; Luke, Timothy J.; Mitchell, David B.; Peels, Rik; Pendrous, Rosina; Sarrazin, Samuel; Schauer, Jacob M.; Specker, Eva; Tran, Ulrich S.; Vranka, Marek A.; Wicherts, Jelte M.; Yoshimura, Naoto; Zwaan, Rolf A.; van Ravenzwaaij, Don
• Is Part Of: Royal Society open science. Volume 10:Issue 2(2023)
• Publication Date: 2023-02-01
• Subjects: replication -- study selection -- consensus; Science -- Periodicals; 500
• DOI: 10.1098/rsos.210586
• Languages: English
• ISSNs: 2054-5703

Abstract

Abstract : Increased execution of replication studies contributes to the effort to restore credibility of empirical research. However, a second generation of problems arises: the number of potential replication targets is at a serious mismatch with available resources. Given limited resources, replication target selection should be well-justified, systematic and transparently communicated. At present the discussion on what to consider when selecting a replication target is limited to theoretical discussion, self-reported justifications and a few formalized suggestions. In this Registered Report, we proposed a study involving the scientific community to create a list of considerations for consultation when selecting a replication target in psychology. We employed a modified Delphi approach. First, we constructed a preliminary list of considerations. Second, we surveyed psychologists who previously selected a replication target with regards to their considerations. Third, we incorporated the results into the preliminary list of considerations and sent the updated list to a group of individuals knowledgeable about concerns regarding replication target selection. Over the course of several rounds, we established consensus regarding what to consider when selecting a replication target. The resulting checklist can be used for transparently communicating the rationale for selecting studies for replication.