An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014–2017). Issue 2 (19th February 2020)

• Record Type: Journal Article
• Authors: Hardwicke, Tom E.; Wallach, Joshua D.; Kidwell, Mallory C.; Bendixen, Theiss; Crüwell, Sophia; Ioannidis, John P. A.
• Is Part Of: Royal Society open science. Volume 7:Issue 2(2020)
• Publication Date: 2020-02-19
• Subjects: transparency -- reproducibility -- meta-research -- social sciences -- open science; Science -- Periodicals; 500
• DOI: 10.1098/rsos.190806
• Languages: English
• ISSNs: 2054-5703

Abstract

Abstract : Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.