Artificial intelligence for imaging-based COVID-19 detection: Systematic review comparing added value of AI versus human readers. Issue 145 (December 2021)

• Record Type: Journal Article
• Authors: Kriza, Christine; Amenta, Valeria; Zenié, Alexandre; Panidis, Dimitris; Chassaigne, Hubert; Urbán, Patricia; Holzwarth, Uwe; Sauer, Aisha Vanessa; Reina, Vittorio; Griesinger, Claudius Benedict
• Is Part Of: European journal of radiology. Issue 145(2021)
• Publication Date: 2021-12
• Subjects: Medical radiology -- Periodicals; Radiology -- Periodicals; Radiologie médicale -- Périodiques; Medical radiology; Periodicals; 616.075705
• DOI: 10.1016/j.ejrad.2021.110028
• Languages: English
• ISSNs: 0720-048X

Abstract

Highlights: There may be an added value of AI model supported imaging-based COVID-19 detection. Studies reported comparable or better performance of AI or AI-supported readings. There was lower variability of diagnostic performance for AI than for human readers. Our systematic review shows heterogeneity of data characteristics and risks of bias. There is a variety of applied methodologies and statistical analysis limitations. Abstract: Purpose: A growing number of studies have examined whether Artificial Intelligence (AI) systems can support imaging-based diagnosis of COVID-19-caused pneumonia, including both gains in diagnostic performance and speed. However, what is currently missing is a combined appreciation of studies comparing human readers and AI. Methods: We followed PRISMA-DTA guidelines for our systematic review, searching EMBASE, PUBMED and Scopus databases. To gain insights into the potential value of AI methods, we focused on studies comparing the performance of human readers versus AI models or versus AI-supported human readings. Results: Our search identified 1270 studies, of which 12 fulfilled specific selection criteria. Concerning diagnostic performance, in testing datasets reported sensitivity was 42–100% (human readers, n = 9 studies), 60–95% (AI systems, n = 10) and 81–98% (AI-supported readers, n = 3), whilst reported specificity was 26–100% (human readers, n = 8), 61–96% (AI systems, n = 10) and 78–99% (AI-supported readings, n = 2). One study highlighted the potential of AI-supported readings for the assessment of lung lesion burden changes, whilst two studies indicated potential time savings for detection with AI. Conclusions: Our review indicates that AI systems or AI-supported human readings show less performance variability (interquartile range) in general, and may support the differentiation of COVID-19 pneumonia from other forms of pneumonia when used in high-prevalence and symptomatic populations. However, inconsistencies related to study design, reporting of data, areas of risk of bias, as well as limitations of statistical analyses complicate clear conclusions. We therefore support efforts for developing critical elements of study design when assessing the value of AI for diagnostic imaging.