NIMG-38. MEASURING ADHERENCE TO TRIPOD OF ARTIFICIAL INTELLIGENCE PAPERS IN THE GLIOMA SEGMENTATION. (12th November 2021)

• Record Type: Journal Article
• Authors: Tillmanns, Niklas; Lum, Avery; Brim, W R; Subramanian, Harry; Lin, Ming; Bousabarah, Khaled; Malhotra, Ajay; cui, Jin; Brackett, Alexandria; Payabvash, Sam; Ikuta, Ichiro; Johnson, Michele; Turowski, Bernd; Aboian, Mariam
• Is Part Of: Neuro-oncology. Volume 23: Supplement 6(2021)
• Page Start: vi137
• Page End: vi137
• Publication Date: 2021-11-12
• Subjects: Brain Neoplasms -- Periodicals; Brain -- Tumors -- Periodicals; Brain -- Cancer -- Periodicals; Nervous system -- Cancer -- Periodicals; 616.99481
• DOI: 10.1093/neuonc/noab196.537
• Languages: English
• ISSNs: 1522-8517

Abstract

Abstract: PURPOSE: Generalizability, reproducibility and objectivity are critical elements that need to be considered when translating machine learning models into clinical practice. While a large body of literature has been published on machine learning methods for segmentation of brain tumors, a systematic evaluation of paper quality and reproducibility has not been done. We investigated the use of "Transparent Reporting of studies on prediction models for Individual Prognosis Or Diagnosis" (TRIPOD) items, among papers published in this relatively new and growing field. METHODS: According to PRISMA a literature review was performed on four databases, Ovid Embase, Ovid MEDLINE, Cochrane trials (CENTRAL) and Web of science core-collection first in October 2020 and a second time in February 2021. Keywords and controlled vocabulary included artificial intelligence, machine learning, deep learning, radiomics, magnetic resonance imaging, glioma, and glioblastoma. The publications were assessed in order to the TRIPOD items. RESULTS: 37 publications from our database search were screened in TRIPOD and yielded an average score of 12.08 with the maximum score being 16 and the minimum score 7. The best scoring item was interpretation (item 19) where all papers scored a point. The lowest scoring items were the title, the abstract, risk groups and the model performance (items number 1, 2, 11 and 16), where no paper scored a point. Less than 1% of the papers discussed the problem of missing data (item 9) and the funding of research (item 22). CONCLUSION: TRIPOD analysis showed that a majority of the papers do not score high on critical elements that allow reproducibility, translation, and objectivity of research. An average score of 12.08 (40%) indicates that the publications usually achieve a relatively low score. The categories that were consistently poorly described include the ML network description, measuring model performance, title details and inclusion of information into the abstract.