The distribution of probability values in medical abstracts: an observational study. (December 2015)

• Record Type: Journal Article
• Authors: Ginsel, Bastiaan; Aggarwal, Abhinav; Xuan, Wei; Harris, Ian
• Is Part Of: BMC research notes. Volume 8:Number 1(2015)
• Page Start: 1
• Page End: 5
• Publication Date: 2015-12
• Subjects: p-hacking -- Publication bias -- p value; Medicine -- Periodicals; Biology -- Periodicals; 610.5
• DOI: 10.1186/s13104-015-1691-x
• Languages: English
• ISSNs: 1756-0500

Abstract

Abstract Background A relatively high incidence of p values immediately below 0.05 (such as 0.047 or 0.04) compared to p values immediately above 0.05 (such as 0.051 or 0.06) has been noticed anecdotally in published medical abstracts. If p values immediately below 0.05 are over-represented, such a distribution may reflect the true underlying distribution of p values or may be due to error (a false distribution). If due to error, a consistent over-representation of p values immediately below 0.05 would be a systematic error due either to publication bias or (overt or inadvertent) bias within studies. Methods We searched the Medline 2012 database to identify abstracts containing a p value. Two thousand abstracts out of 80, 649 abstracts were randomly selected. Two independent researchers extracted all p values. The p values were plotted and compared to a predicted curve. Chi square test was used to test assumptions and significance was set at 0.05. Results 2798 p value ranges and 3236 exact p values were reported. 4973 of these (82 %) were significant (<0.05). There was an over-representation of p values immediately below 0.05 (between 0.01 and 0.049) compared to those immediately above 0.05 (between 0.05 and 0.1) (p = 0.001). Conclusion The distribution of p values in reported medical abstracts provides evidence for systematic error in the reporting of p values. This may be due to publication bias, methodological errors (underpowering, selective reporting and selective analyses) or fraud.