Automatic estimation of ulcerative colitis severity from endoscopy videos using ordinal multi-instance learning. Issue 4 (4th July 2022)

• Record Type: Journal Article
• Authors: Schwab, Evan; Cula, Gabriela Oana; Standish, Kristopher; Yip, Stephen S. F.; Stojmirovic, Aleksandar; Ghanem, Louis; Chehoud, Christel
• Is Part Of: Computer methods in biomechanics and biomedical engineering. Volume 10:Issue 4(2022)
• Page Start: 425
• Page End: 433
• Publication Date: 2022-07-04
• Subjects: IBD -- Ulcerative Colitis -- endoscopy video -- severity estimation -- weak supervision -- multi-instance learning -- ordinal classification -- deep learning -- convolutional neural networks; Imaging systems in biology -- Periodicals; Imaging systems in medicine -- Periodicals; Biomechanics -- Data processing -- Periodicals; Biomedical engineering -- Periodicals; 616.0757
• DOI: 10.1080/21681163.2021.1997644
• Languages: English
• ISSNs: 2168-1163

Abstract

ABSTRACT: Ulcerative colitis (UC) is an inflammatory bowel disease characterised by inflammation of the large intestine. UC severity is often represented by the Mayo Endoscopic Subscore (MES) which quantifies disease activity from endoscopy videos. In clinical trials, an endoscopy video is assigned an MES based upon the most severe inflammation observed in the video. For this reason, severe inflammation spread throughout the colon will receive the same MES as an otherwise healthy colon with a small localized inflammation. Therefore, the extent of disease activity throughout the colon may not be completely captured by the MES. In this work, we aim to automatically estimate MES for each frame in an endoscopy video to provide a higher resolution assessment of UC. Because annotating severity at the frame-level is highly expensive and labour-intensive, we propose a novel weakly supervised, ordinal classification method to estimate frame severity from video labels alone. Using clinical trial data, we demonstrate that our models achieve substantial agreement with ground truth labels, comparable to the agreement between expert clinicians. These findings indicate that our framework could serve as a foundation for novel clinical endpoints, based on a more localised scoring system, to better evaluate UC drug efficacy in clinical trials.