Constrained multiple instance learning for ulcerative colitis prediction using histological images. (September 2022)

• Record Type: Journal Article
• Authors: del Amor, Rocío; Meseguer, Pablo; Parigi, Tommaso Lorenzo; Villanacci, Vincenzo; Colomer, Adrián; Launet, Laëtitia; Bazarova, Alina; Tontini, Gian Eugenio; Bisschops, Raf; de Hertogh, Gert; Ferraz, Jose G.; Götz, Martin; Gui, Xianyong; Hayee, Bu'Hussain; Lazarev, Mark; Panaccione, Remo; Parra-Blanco, Adolfo; Bhandari, Pradeep; Pastorelli, Luca; Rath, Timo; Røyset, Elin Synnøve; Vieth, Michael; Zardo, Davide; Grisan, Enrico; Ghosh, Subrata; Iacucci, Marietta; Naranjo, Valery
• Is Part Of: Computer methods and programs in biomedicine. Volume 224(2022)
• Publication Date: 2022-09
• Subjects: Histologic remission -- Location constraints -- Neutrophils -- Attention-embedding weights -- Ulcerative colitis; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2022.107012
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: Colon and rectum histological images are used for the first time to develop an end-to-end automatic system able to predict ulcerative colitis activity based on neutrophil detection using WSI, which is a diagnostic challenge. A novel constrained formulation for multiple instance learning (MIL) integrating an auxiliary term that focuses on the key features for the classification is proposed. A new attention weight for embedding-level MIL, which enlarges the relevance of the positive instances is presented. With the use of these coefficients, the final representation of the bag is highly informative for the bag-level classifier. We benchmark the proposed model against the relevant body of literature on PICASSO-MIL, a multi-center database composed of a large cohort of biopsies collected and digitalized in 7 centers in the UK, Germany, Belgium, Italy, Canada and USA. Comprehensive experiments demonstrate the superior performance of our model. By simply incorporating information about the neutrophil location during the training, we found improvements of nearly 10% for bag-level classification compared to the most relevant MIL methods. The class activation maps extracted are directly in line with the clinician's opinion since the highlighted regions of the histological images correspond to the interesting areas in which the experts focus for Ulcerative Colitis diagnosis. Abstract: Background and Objective: Ulcerative colitis (UC) is an inflammatory bowel disease (IBD) affecting the colon and the rectum characterized by a remitting-relapsing course. To detect mucosal inflammation associated with UC, histology is considered the most stringent criteria. In turn, histologic remission (HR) correlates with improved clinical outcomes and has been recently recognized as a desirable treatment target. The leading biomarker for assessing histologic remission is the presence or absence of neutrophils. Therefore, the finding of this cell in specific colon structures indicates that the patient has UC activity. However, no previous studies based on deep learning have been developed to identify UC based on neutrophils detection using whole-slide images (WSI). Methods: The methodological core of this work is a novel multiple instance learning (MIL) framework with location constraints able to determine the presence of UC activity using WSI. In particular, we put forward an effective way to introduce constraints about positive instances to effectively explore additional weakly supervised information that is easy to obtain and enjoy a significant boost to the learning process. In addition, we propose a new weighted embedding to enlarge the relevance of the positive instances. Results: Extensive experiments on a multi-center dataset of colon and rectum WSIs, PICASSO-MIL, demonstrate that using the location information we can improve considerably the results at WSI-level. In comparison with prior MIL settings, our method allows for 10 % improvements in bag-level accuracy. Conclusion : Our model, which introduces a new form of constraints, surpass the results achieved from current state-of-the-art methods that focus on the MIL paradigm. Our method can be applied to other histological concerns where the morphological features determining a positive WSI are tiny and similar to others in the image.