Towards realistic laparoscopic image generation using image-domain translation. (March 2021)

• Record Type: Journal Article
• Authors: Marzullo, Aldo; Moccia, Sara; Catellani, Michele; Calimeri, Francesco; Momi, Elena De
• Is Part Of: Computer methods and programs in biomedicine. Volume 200(2021)
• Publication Date: 2021-03
• Subjects: Generative Adversarial Networks -- Minimally Invasive Surgery -- Image translation -- Data Augmentation; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2020.105834
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: We propose a conditional generative adversarial network for laparoscopic image synthesis The pix2pix model was adapted to translate rough segmentation of surgical instruments and tissues into realistic images Quantitative and human-based evaluations of generated images have been carried out in order to assess the effectiveness of the method Abstract: Background and Objectives Over the last decade, Deep Learning (DL) has revolutionized data analysis in many areas, including medical imaging. However, there is a bottleneck in the advancement of DL in the surgery field, which can be seen in a shortage of large-scale data, which in turn may be attributed to the lack of a structured and standardized methodology for storing and analyzing surgical images in clinical centres. Furthermore, accurate annotations manually added are expensive and time consuming. A great help can come from the synthesis of artificial images; in this context, in the latest years, the use of Generative Adversarial Neural Networks (GANs) achieved promising results in obtaining photo-realistic images. Methods In this study, a method for Minimally Invasive Surgery (MIS) image synthesis is proposed. To this aim, the generative adversarial network pix2pix is trained to generate paired annotated MIS images by transforming rough segmentation of surgical instruments and tissues into realistic images. An additional regularization term was added to the original optimization problem, in order to enhance realism of surgical tools with respect to the background. Results Quantitative and qualitative (i.e., human-based) evaluations of generated images have been carried out in order to assess the effectiveness of the method. Conclusions Experimental results show that the proposed method is actually able to translate MIS segmentations to realistic MIS images, which can in turn be used to augment existing data sets and help at overcoming the lack of useful images; this allows physicians and algorithms to take advantage from new annotated instances for their training.