Fact or fiction: reducing the proportion and impact of false positives. Issue 7 (27th November 2017)

• Record Type: Journal Article
• Authors: Stahl, D.; Pickles, A.
• Is Part Of: Psychological medicine. Volume 48:Issue 7(2018)
• Page Start: 1084
• Page End: 1091
• Publication Date: 2017-11-27
• Subjects: Bayes' factor, -- cross-validation, -- external validation, -- false positives, -- meta-analyses, -- replication crisis, -- researcher degrees-of-freedom, -- statistical learning; Psychiatry -- Periodicals; Medicine and psychology -- Periodicals; Clinical psychology -- Periodicals; 616.89
• DOI: 10.1017/S003329171700294X
• Languages: English
• ISSNs: 0033-2917

Abstract

Abstract: False positive findings in science are inevitable, but are they particularly common in psychology and psychiatry? The evidence that we review suggests that while not restricted to our field, the problem is acute. We describe the concept of researcher 'degrees-of-freedom' to explain how many false-positive findings arise, and how the various strategies of registration, pre-specification, and reporting standards that are being adopted both reduce and make these visible. We review possible benefits and harms of proposed statistical solutions, from tougher requirements for significance, to Bayesian and machine learning approaches to analysis. Finally we consider the organisation and methods for replication and systematic review in psychology and psychiatry.