Deep learning-based automated detection of pulmonary embolism on CT pulmonary angiograms: No significant effects on report communication times and patient turnaround in the emergency department nine months after technical implementation. Issue 141 (August 2021)

• Record Type: Journal Article
• Authors: Schmuelling, Lena; Franzeck, Fabian C.; Nickel, Christian H.; Mansella, Gregory; Bingisser, Roland; Schmidt, Noemi; Stieltjes, Bram; Bremerich, Jens; Sauter, Alexander W.; Weikert, Thomas; Sommer, Gregor
• Is Part Of: European journal of radiology. Issue 141(2021)
• Publication Date: 2021-08
• Subjects: ADMIRE Advanced Modeled Iterative Reconstruction -- ANOVA Analysis of Variance -- AI Artificial Intelligence -- BL Baseline -- CAD Computer-assisted detection -- CT Computed Tomography -- CTPA CT Pulmonary Angiogram -- DICOM Digital Imaging and Communication in Medicine -- DL Deep Learning -- ED Emergency Department -- ENS Electronic Notification System -- FN False Negative -- FP False Positive -- HTTPS Hypertext Transfer Protocol Secure -- IR Iterative Reconstruction -- IRB Institutional Review Board -- LM Light Messenger -- NPV Negative Predictive Value -- PACS Picture Archiving and Communication System -- PE Pulmonary Embolism -- PE+ Positive for Pulmonary Embolism -- PE− Negative for Pulmonary Embolism -- PPV Positive Predictive Value -- RCT Report Communication Time -- RIS Radiology Information System -- RRT Report Reading Time -- SAFIRE Sinogram Affirmed Reconstruction -- SD Standard Deviation -- SOP Standard Operating Procedure -- TAT Turnaround Time -- TN True Negative -- TP True Positive -- TTA Time to Anticoagulation; Pulmonary embolism -- Computed tomography pulmonary angiograms -- Artificial intelligence -- Clinical workflow -- Communication -- Emergency department; Medical radiology -- Periodicals; Radiology -- Periodicals; Radiologie médicale -- Périodiques; Medical radiology; Periodicals; 616.075705
• DOI: 10.1016/j.ejrad.2021.109816
• Languages: English
• ISSNs: 0720-048X

Abstract

Highlights: We used an electronic notification system and deep learning-assisted finding detection for pulmonary embolism in CTPAs. The used DL-algorithm showed a good diagnostic accuracy (82.2 % positive predictive value, 94.1 % negative predictive value) for the detection of pulmonary embolism in CTPAs on the large, consecutive clinical dataset. The systems did not significantly reduce report communication times in a large patient sample nine months after clinical implementation. Furthermore, turnaround times of patients diagnosed with pulmonary embolism in the emergency department were not significantly reduced. This underpins the importance of a structured clinical introduction and implementation of DL tools for having a significant impact on clinical outcomes. Abstract: Objectives: Rapid communication of CT exams positive for pulmonary embolism (PE) is crucial for timely initiation of anticoagulation and patient outcome. It is unknown if deep learning automated detection of PE on CT Pulmonary Angiograms (CTPA) in combination with worklist prioritization and an electronic notification system (ENS) can improve communication times and patient turnaround in the Emergency Department (ED). Methods: In 01/2019, an ENS allowing direct communication between radiology and ED was installed. Starting in 10/2019, CTPAs were processed by a deep learning (DL)-powered algorithm for detection of PE. CTPAs acquired between 04/2018 and 06/2020 (n = 1808) were analysed. To assess the impact of the ENS and the DL-algorithm, radiology report reading times (RRT), radiology report communication time (RCT), time to anticoagulation (TTA), and patient turnaround times (TAT) in the ED were compared for three consecutive time periods. Performance measures of the algorithm were calculated on a per exam level (sensitivity, specificity, PPV, NPV, F1-score), with written reports and exam review as ground truth. Results: Sensitivity of the algorithm was 79.6 % (95 %CI:70.8−87.2%), specificity 95.0 % (95 %CI:92.0−97.1%), PPV 82.2 % (95 %CI:73.9−88.3), and NPV 94.1 % (95 %CI:91.4–96 %). There was no statistically significant reduction of any of the observed times (RRT, RCT, TTA, TAT). Conclusion: DL-assisted detection of PE in CTPAs and ENS-assisted communication of results to referring physicians technically work. However, the mere clinical introduction of these tools, even if they exhibit a good performance, is not sufficient to achieve significant effects on clinical performance measures.