Deep multimodal predictome for studying mental disorders. Issue 2 (15th September 2022)

• Record Type: Journal Article
• Authors: Rahaman, Md Abdur; Chen, Jiayu; Fu, Zening; Lewis, Noah; Iraji, Armin; van Erp, Theo G. M.; Calhoun, Vince D.
• Is Part Of: Human brain mapping. Volume 44:Issue 2(2023)
• Page Start: 509
• Page End: 522
• Publication Date: 2022-09-15
• Subjects: functional network connectivity -- multimodal deep learning -- resting‐state functional and structural MRI -- saliency -- schizophrenia classification -- single nucleotide polymorphism; Brain mapping -- Periodicals; 611.81
• DOI: 10.1002/hbm.26077
• Languages: English
• ISSNs: 1065-9471

Abstract

Abstract: Characterizing neuropsychiatric disorders is challenging due to heterogeneity in the population. We propose combining structural and functional neuroimaging and genomic data in a multimodal classification framework to leverage their complementary information. Our objectives are two‐fold (i) to improve the classification of disorders and (ii) to introspect the concepts learned to explore underlying neural and biological mechanisms linked to mental disorders. Previous multimodal studies have focused on naïve neural networks, mostly perceptron, to learn modality‐wise features and often assume equal contribution from each modality. Our focus is on the development of neural networks for feature learning and implementing an adaptive control unit for the fusion phase. Our mid fusion with attention model includes a multilayer feed‐forward network, an autoencoder, a bi‐directional long short‐term memory unit with attention as the features extractor, and a linear attention module for controlling modality‐specific influence. The proposed model acquired 92% ( p < .0001) accuracy in schizophrenia prediction, outperforming several other state‐of‐the‐art models applied to unimodal or multimodal data. Post hoc feature analyses uncovered critical neural features and genes/biological pathways associated with schizophrenia. The proposed model effectively combines multimodal neuroimaging and genomics data for predicting mental disorders. Interpreting salient features identified by the model may advance our understanding of their underlying etiological mechanisms. Abstract : Multimodal deep learning of imaging genetics data for characterizing mental disorders. The model processes multisource physiological data. It boosts the schizophrenia classifications accuracy by a margin compared with the existing model. The interpretation provides a handful of neurological ad genomics features that explains the underlying mechanism of the disease.