Memory-efficient GAN-based domain translation of high resolution 3D medical images. (December 2020)

• Record Type: Journal Article
• Authors: Uzunova, Hristina; Ehrhardt, Jan; Handels, Heinz
• Is Part Of: Computerized medical imaging and graphics. Volume 86(2020)
• Publication Date: 2020-12
• Subjects: Multi-scale GAN -- Memory-efficient -- High resolution -- 3D medical images; Diagnostic imaging -- Periodicals; Imaging systems in medicine -- Periodicals; Diagnosis, Radioscopic -- Data processing -- Periodicals; Diagnostic Imaging -- Periodicals; Imagerie pour le diagnostic -- Périodiques; Diagnostic imaging; Periodicals; Electronic journals; Electronic journals; 616.0754
• DOI: 10.1016/j.compmedimag.2020.101801
• Languages: English
• ISSNs: 0895-6111

Abstract

Highlights: Generative adversarial networks (GANs) are rarely applied on 3D medical images due to their immense computational demand. The present work proposes a multi-scale patch-based GAN approach for large 3D medical images. Constant memory demand for arbitrary image size, yet no patch artifacts. High quality of images due to the multiscale approach. Tested for exceptionally large images of size up to 512 × 512 × 512. Abstract: Generative adversarial networks (GANs) are currently rarely applied on 3D medical images of large size, due to their immense computational demand. The present work proposes a multi-scale patch-based GAN approach for establishing unpaired domain translation by generating 3D medical image volumes of high resolution in a memory-efficient way. The key idea to enable memory-efficient image generation is to first generate a low-resolution version of the image followed by the generation of patches of constant sizes but successively growing resolutions. To avoid patch artifacts and incorporate global information, the patch generation is conditioned on patches from previous resolution scales. Those multi-scale GANs are trained to generate realistically looking images from image sketches in order to perform an unpaired domain translation. This allows to preserve the topology of the test data and generate the appearance of the training domain data. The evaluation of the domain translation scenarios is performed on brain MRIs of size 155 × 240 × 240 and thorax CTs of size up to 512 3 . Compared to common patch-based approaches, the multi-resolution scheme enables better image quality and prevents patch artifacts. Also, it ensures constant GPU memory demand independent from the image size, allowing for the generation of arbitrarily large images.