NEgatiVE results in Radiomics research (NEVER): A meta-research study of publication bias in leading radiology journals. Issue 163 (June 2023)

• Record Type: Journal Article
• Authors: Kocak, Burak; Bulut, Elif; Bayrak, Osman Nuri; Okumus, Ahmet Arda; Altun, Omer; Borekci Arvas, Zeynep; Kavukoglu, Irem
• Is Part Of: European journal of radiology. Issue 163(2023)
• Publication Date: 2023-06
• Subjects: Radiomics -- Texture analysis -- Systematic review -- Meta-research -- Publication bias -- Negative results; Medical radiology -- Periodicals; Radiology -- Periodicals; Radiologie médicale -- Périodiques; Medical radiology; Periodicals; 616.075705
• DOI: 10.1016/j.ejrad.2023.110830
• Languages: English
• ISSNs: 0720-048X

Abstract

Highlights: Based on a priori power analysis, a random sample of 149 publications was reviewed. Nearly half of the publications did not compare their radiomic method to a non-radiomic method. Leading clinical radiology journals have a strong bias toward publishing positive results. Almost never do these journals publish negative results. Abstract: Purpose: The purpose of this study was to conduct a meta -research of radiomics-related articles for the publication of negative results, with a focus on the leading clinical radiology journals due to their purportedly high editorial standards. Methods: A literature search was performed in PubMed to identify original research studies on radiomics (last search date: August 16th, 2022). The search was restricted to studies published in Q1 clinical radiology journals indexed by Scopus and Web of Science. Following an a priori power analysis based on our null hypothesis, a random sampling of the published literature was conducted. Besides the six baseline study characteristics, a total of three items about publication bias were evaluated. Agreement between raters was analyzed. Disagreements were resolved through consensus. Statistical synthesis of the qualitative evaluations was presented. Results: Following a priori power analysis, we included a random sample of 149 publications in this study. Most of the publications were retrospective (95%; 142/149), based on private data (91%; 136/149), centered on a single institution (75%; 111/149), and lacked external validation (81%; 121/149). Slightly fewer than half (44%; 66/149) made no comparison to non-radiomic approaches. Overall, only one study (1%; 1/149) reported negative results for radiomics, yielding a statistically significant binomial test ( p < 0.0001). Conclusion: The top clinical radiology journals almost never publish negative results, having a strong bias toward publishing positive results. Almost half of the publications did not even compare their approach with a non-radiomic method.