The Complexities of Evaluating the Exposome in Psychiatry: A Data-Driven Illustration of Challenges and Some Propositions for Amendments. (30th August 2018)

• Record Type: Journal Article
• Authors: Guloksuz, Sinan; Rutten, Bart P F; Pries, Lotta-Katrin; ten Have, Margreet; de Graaf, Ron; van Dorsselaer, Saskia; Klingenberg, Boris; van Os, Jim; Ioannidis, John P A
• Is Part Of: Schizophrenia bulletin. Volume 44:Number 6(2018:Nov.)
• Page Start: 1175
• Page End: 1179
• Publication Date: 2018-08-30
• Subjects: Psychosis -- Schizophrenia -- Environment -- Cannabis -- Childhood trauma -- Urbanicity -- Reproducibility -- Causality -- Bias; Schizophrenia -- Periodicals; Schizophrenia -- Research -- Periodicals; 616.898005
• DOI: 10.1093/schbul/sby118
• Languages: English
• ISSNs: 0586-7614

Abstract

Abstract: Identifying modifiable factors through environmental research may improve mental health outcomes. However, several challenges need to be addressed to optimize the chances of success. By analyzing the Netherlands Mental Health Survey and Incidence Study-2 data, we provide a data-driven illustration of how closely connected the exposures and the mental health outcomes are and how model and variable specifications produce "vibration of effects" (variation of results under multiple different model specifications). Interdependence of exposures is the rule rather than the exception. Therefore, exposure-wide systematic approaches are needed to separate genuine strong signals from selective reporting and dissect sources of heterogeneity. Pre-registration of protocols and analytical plans is still uncommon in environmental research. Different studies often present very different models, including different variables, despite examining the same outcome, even if consistent sets of variables and definitions are available. For datasets that are already collected (and often already analyzed), the exploratory nature of the work should be disclosed. Exploratory analysis should be separated from prospective confirmatory research with truly pre-specified analysis plans. In the era of big-data, where very low P values for trivial effects are detected, several safeguards may be considered to improve inferences, eg, lowering P -value thresholds, prioritizing effect sizes over significance, analyzing pre-specified falsification endpoints, and embracing alternative approaches like false discovery rates and Bayesian methods. Any claims for causality should be cautious and preferably avoided, until intervention effects have been validated. We hope the propositions for amendments presented here may help with meeting these pressing challenges.