Bi-MGAN: Bidirectional T1-to-T2 MRI images prediction using multi-generative multi-adversarial nets. (September 2022)

• Record Type: Journal Article
• Authors: Xu, Liming; Zhang, He; Song, Lanyu; Lei, Yanrong
• Is Part Of: Biomedical signal processing and control. Volume 78(2022)
• Publication Date: 2022-09
• Subjects: Bidirectional prediction -- Generative adversarial networks -- Pathological invariance -- Multi-feature fusion; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.103994
• Languages: English
• ISSNs: 1746-8094

Abstract

Abstract: Magnetic Resonance Imaging (MRI) is a widely-used detection technology both in clinical treatment and image-guide therapy, and it can yield multi-modal medical images. However, MRI technique requires expensive costs and long-time process, which causes extra financial burden and low throughput. Therefore, it is necessary to estimate desired modality from acquired modality without real acquisition, which can effectively decrease financial cost and indirectly improve throughput. Considering that current unidirectional methods can only predict one modality fixedly, we propose a bidirectional prediction method by using multi-generative multi-adversarial nets (Bi-MGAN) to predict desired modality from acquired modality in paired and unpaired fashion. Our method simultaneously learns two nonlinear mappings between T1-weighted MRI and T2-weighted MRI image. Within it, pathological label is introduced to ensure invariance of pathological information. Then, deep and handcrafted features are fused to preserve consistency of context feature and anatomical structure. Besides, spectral normalization is utilized to control the performance of discriminator and accelerate learning process, and local regularization is used to reduce generalization error of predictors. Experimental results show that our Bi-MGAN has preserved pathological feature and anatomical structure comparing with other state-of-the-art methods. It gains significant increments on paired dataset and obtains competitive results on unpaired dataset. Specifically, it achieves average increment of 57.1% SSIM, 47.1% FSIM and 50.0% MSIM on paired SPLP dataset, and obtains 9.1% SSIM, 3.6% FSIM and 9.6% MSIM on unpaired Brain dataset comparing with others. Highlights: We propose bidirectional prediction approach to estimate desired modal MRI images from acquired MRI images in paired and unpaired fashion, which is the first end-to-end bidirectional method for multi-modal MRI images prediction. We introduce auxiliary label information to suppress additionally generated content beyond original images, and prevent potential risk that pathological information of original image may be changed and further ensure the pathological invariance. We adopt spectral normalization to control the performance of discriminator and accelerate training. Within it, deep and hand-crafted features are fused to constrain feature generation. Extensive experiments on two public and one private datasets show that our Bi-MGAN works better than recently state-of-the-art prediction methods in paired and unpaired fashion.