A video based benchmark data set (ENDOTEST) to evaluate computer-aided polyp detection systems. (2nd November 2022)

• Record Type: Journal Article
• Authors: Fitting, Daniel; Krenzer, Adrian; Troya, Joel; Banck, Michael; Sudarevic, Boban; Brand, Markus; Böck, Wolfgang; Zoller, Wolfram G.; Rösch, Thomas; Puppe, Frank; Meining, Alexander; Hann, Alexander
• Is Part Of: Scandinavian journal of gastroenterology. Volume 57:Number 11(2022)
• Page Start: 1397
• Page End: 1403
• Publication Date: 2022-11-02
• Subjects: Colonoscopy -- polyp -- artificial intelligence -- deep learning -- CADe; Gastroenterology -- Periodicals; Digestive organs -- Diseases -- Periodicals; 616.33
• DOI: 10.1080/00365521.2022.2085059
• Languages: English
• ISSNs: 0036-5521

Abstract

Abstract: Background and aims: Computer-aided polyp detection (CADe) may become a standard for polyp detection during colonoscopy. Several systems are already commercially available. We report on a video-based benchmark technique for the first preclinical assessment of such systems before comparative randomized trials are to be undertaken. Additionally, we compare a commercially available CADe system with our newly developed one. Methods: ENDOTEST consisted in the combination of two datasets. The validation dataset contained 48 video-snippets with 22, 856 manually annotated images of which 53.2% contained polyps. The performance dataset contained 10 full-length screening colonoscopies with 230, 898 manually annotated images of which 15.8% contained a polyp. Assessment parameters were accuracy for polyp detection and time delay to first polyp detection after polyp appearance (FDT). Two CADe systems were assessed: a commercial CADe system (GI-Genius, Medtronic), and a self-developed new system (ENDOMIND). The latter being a convolutional neuronal network trained on 194, 983 manually labeled images extracted from colonoscopy videos recorded in mainly six different gastroenterologic practices. Results: On the ENDOTEST, both CADe systems detected all polyps in at least one image. The per-frame sensitivity and specificity in full colonoscopies was 48.1% and 93.7%, respectively for GI-Genius; and 54% and 92.7%, respectively for ENDOMIND. Median FDT of ENDOMIND with 217 ms (Inter-Quartile Range(IQR)8–1533) was significantly faster than GI-Genius with 1050 ms (IQR 358–2767, p = 0.003). Conclusions: Our benchmark ENDOTEST may be helpful for preclinical testing of new CADe devices. There seems to be a correlation between a shorter FDT with a higher sensitivity and a lower specificity for polyp detection.