Beyond psychology: prevalence of p value and confidence interval misinterpretation across different fields. (2020)

• Record Type: Journal Article
• Authors: Lyu, Xiao-Kang; Xu, Yuepei; Zhao, Xiao-Fan; Zuo, Xi-Nian; Hu, Chuan-Peng
• Is Part Of: Journal of Pacific Rim psychology. Volume 14(2020)
• Publication Date: 2020
• Subjects: p value -- confidence intervals -- statistical training -- meta-research; Psychology -- Oceania -- Periodicals; Psychology -- Islands of the Pacific -- Periodicals; Psychology -- Periodicals; 150.99
• DOI: 10.1017/prp.2019.28
• Languages: English
• ISSNs: 1834-4909

Abstract

P values and confidence intervals (CIs) are the most widely used statistical indices in scientific literature. Several surveys have revealed that these two indices are generally misunderstood. However, existing surveys on this subject fall under psychology and biomedical research, and data from other disciplines are rare. Moreover, the confidence of researchers when constructing judgments remains unclear. To fill this research gap, we surveyed 1, 479 researchers and students from different fields in China. Results reveal that for significant (i.e., p < .05, CI does not include zero) and non-significant (i.e., p > .05, CI includes zero) conditions, most respondents, regardless of academic degrees, research fields and stages of career, could not interpret p values and CIs accurately. Moreover, the majority were confident about their (inaccurate) judgements (seeosf.io/mcu9q/ for raw data, materials, and supplementary analyses). Therefore, as misinterpretations of p values and CIs prevail in the whole scientific community, there is a need for better statistical training in science.