Characterizing functional brain networks via Spatio-Temporal Attention 4D Convolutional Neural Networks (STA-4DCNNs). (January 2023)

• Record Type: Journal Article
• Authors: Jiang, Xi; Yan, Jiadong; Zhao, Yu; Jiang, Mingxin; Chen, Yuzhong; Zhou, Jingchao; Xiao, Zhenxiang; Wang, Zifan; Zhang, Rong; Becker, Benjamin; Zhu, Dajiang; Kendrick, Keith M.; Liu, Tianming
• Is Part Of: Neural networks. Volume 158(2023)
• Page Start: 99
• Page End: 110
• Publication Date: 2023-01
• Subjects: Four-Dimensional Convolutional Neural Network -- Functional brain network -- Functional MRI -- Individualized -- Spatio-temporal pattern; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2022.11.004
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Characterizing individualized spatio-temporal patterns of functional brain networks (FBNs) via functional magnetic resonance imaging (fMRI) provides a foundation for understanding complex brain function. Although previous studies have achieved promising performances based on either shallow or deep learning models, there is still much space to improve the accuracy of spatio-temporal pattern characterization of FBNs by optimally integrating the four-dimensional (4D) features of fMRI. In this study, we introduce a novel Spatio-Temporal Attention 4D Convolutional Neural Network (STA-4DCNN) model to characterize individualized spatio-temporal patterns of FBNs. Particularly, STA-4DCNN is composed of two subnetworks, in which the first Spatial Attention 4D CNN (SA-4DCNN) models the spatio-temporal features of 4D fMRI data and then characterizes the spatial pattern of FBNs, and the second Temporal Guided Attention Network (T-GANet) further characterizes the temporal pattern of FBNs under the guidance of the spatial pattern together with 4D fMRI data. We evaluate the proposed STA-4DCNN on seven different task fMRI and one resting state fMRI datasets from the publicly released Human Connectome Project. The experimental results demonstrate that STA-4DCNN has superior ability and generalizability in characterizing individualized spatio-temporal patterns of FBNs when compared to other state-of-the-art models. We further apply STA-4DCNN on another independent ABIDE I resting state fMRI dataset including both autism spectrum disorder (ASD) and typical developing (TD) subjects, and successfully identify abnormal spatio-temporal patterns of FBNs in ASD compared to TD. In general, STA-4DCNN provides a powerful tool for FBN characterization and for clinical applications on brain disease characterization at the individual level. Highlights: 4D convolution is adopted to fully characterize 4D nature of fMRI data. Attention copy operation is adopted to improve U-net learning ability. Guided attention helps jointly model spatio-temporal patterns of brain networks. The proposed model is effective in characterizing abnormal patterns in ASD.