The P value plot does not provide evidence against air pollution hazards. Issue 2 (18th April 2022)

• Record Type: Journal Article
• Authors: Hicks, Daniel J.
• Is Part Of: Environmental epidemiology. Volume 6:Issue 2(2022)
• Page Start: e198
• Publication Date: 2022-04-18
• Subjects: P value plot -- Simulation methods -- P hacking -- Air pollution; Epidemiology -- Periodicals; Epidemiology -- Environmental aspects -- Periodicals; 614.4
• DOI: 10.1097/EE9.0000000000000198
• Languages: English
• ISSNs: 2474-7882

Abstract

Abstract : Background: A number of papers by Young and collaborators have criticized epidemiological studies and meta-analyses of air pollution hazards using a graphical method that the authors call a P value plot, claiming to find zero effects, heterogeneity, and P hacking. However, the P value plot method has not been validated in a peer-reviewed publication. The aim of this study was to investigate the statistical and evidentiary properties of this method. Methods: A simulation was developed to create studies and meta-analyses with known real effects δ, integrating two quantifiable conceptions of evidence from the philosophy of science literature. The simulation and analysis is publicly available and automatically reproduced. Results: In this simulation, the plot did not provide evidence for heterogeneity or P hacking with respect to any condition. Under the right conditions, the plot can provide evidence of zero effects; but these conditions are not satisfied in any actual use by Young and collaborators. Conclusion: The P value plot does not provide evidence to support the skeptical claims about air pollution hazards made by Young and collaborators.