Effects of a proposal to alter the statistical significance threshold on previously published orthopaedic trauma randomized controlled trials. Issue 11 (November 2019)

• Record Type: Journal Article
• Authors: Johnson, Austin L.; Evans, Sheridan; Checketts, Jake X.; Scott, Jared T.; Wayant, Cole; Johnson, Mark; Norris, Brent; Vassar, Matt
• Is Part Of: Injury. Volume 50:Issue 11(2019)
• Page Start: 1934
• Page End: 1937
• Publication Date: 2019-11
• Subjects: Statistical significance -- P-value -- Orthopaedic -- Orthopaedic trials -- Trauma -- Traumatology -- Randomized controlled trials -- Clinical trials -- Analysis -- Evidence -- Statistics; Wounds and injuries -- Surgery -- Periodicals; Accidents -- Periodicals; Wounds and Injuries -- surgery -- Periodicals; Lésions et blessures -- Chirurgie -- Périodiques; Electronic journals; Electronic journals; 617.1
• DOI: 10.1016/j.injury.2019.08.012
• Languages: English
• ISSNs: 0020-1383

Abstract

Highlights: It has been suggested that we adopt a new significance level by lowering the threshold from .05 to .005. This proposal is meant to serve as a temporizing measure while long-term strategies are developed, tested, and implemented. A common misconception in clinical research is that statistical significance equates to clinical significance. Abstract: Introduction: A recent proposal suggests changing the threshold for statistical significance from a P value of .05 to .005 to minimize bias and increase reproducibility of future studies. P values less than .05 but greater than .005 would be reclassified as "suggestive", whereas P values less than .005 would be considered significant. The present study explores how lowering the P value threshold would affect the interpretation of previously published orthopaedic trauma randomized controlled trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. Methods: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery were screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. All P values for primary endpoints were included from each study. Results: We identified 124 primary endpoints from 48 trials: 39.5% (49/124) of endpoints had a P value less than .05 and 60.5% (75/124) had a P value greater than .05. Overall, 51.0% (25/49) of statistically significant primary endpoints were less than .005, while 49.0% (24/49) would be reclassified as suggestive. Conclusion: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the effect such a proposal on orthopaedic practice.