Patch-based deep multi-modal learning framework for Alzheimer's disease diagnosis using multi-view neuroimaging. (February 2023)

• Record Type: Journal Article
• Authors: Liu, Fangyu; Yuan, Shizhong; Li, Weimin; Xu, Qun; Sheng, Bin
• Is Part Of: Biomedical signal processing and control. Volume 80:Part 2(2023)
• Publication Date: 2023-02
• Subjects: Alzheimer's disease -- Neuroimages -- Discriminative lesion localization -- Convolutional neural network -- Multi-modal fusion; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.104400
• Languages: English
• ISSNs: 1746-8094

Abstract

Abstract: The computer-aided diagnosis contributes to the early detection of mild cognitive impairment (MCI). Though many deep learning methods have achieved favorable performance on Alzheimer's disease (AD) diagnosis tasks, the single-modality method will lead to lopsidedness in modeling disease features. Secondly, existing patch-based methods usually ignore the spatial information between local image patches when modeling the global feature representation of the brain. In addition, existing patch-based methods rely on anatomical landmark detection algorithms to pre-determine informative locations in the brain, which would result in the localization of brain atrophy locations that not only require extensive expert experience but also misses some potential lesion areas. In this paper, we propose a patch-based deep multi-modal learning (PDMML) framework for brain disease diagnosis. Specifically, we design a discriminative location discovery strategy to filter normal regions without prior knowledge. Multimodal imaging features were integrated at the patch level to capture multi-view brain disease representations. The local patches are further jointly learned to prevent the loss of spatial information caused by the direct flattening of the patches. Experimental results on 842 subjects from the ADNI dataset demonstrate that our proposed method excels in discriminative location discovery and brain disease diagnosis. Highlights: Features of the five modality brain images were fused at the patch level. Lesion-sensitive locations in 3D brain images were screened in a data-driven manner. Local patches are further jointly learned to prevent spatial information loss. Multi-modal images provide subjects with multi-view information. Multi-modal features can model subjects more comprehensively.