What Is the Test-Retest Reliability of Common Task-Functional MRI Measures? New Empirical Evidence and a Meta-Analysis. (July 2020)

• Record Type: Journal Article
• Authors: Elliott, Maxwell L.; Knodt, Annchen R.; Ireland, David; Morris, Meriwether L.; Poulton, Richie; Ramrakha, Sandhya; Sison, Maria L.; Moffitt, Terrie E.; Caspi, Avshalom; Hariri, Ahmad R.
• Is Part Of: Psychological science. Volume 31:Number 7(2020)
• Page Start: 792
• Page End: 806
• Publication Date: 2020-07
• Subjects: neuroimaging -- individual differences -- statistical analysis -- cognitive neuroscience; Psychology -- Periodicals; 150.5
• DOI: 10.1177/0956797620916786
• Languages: English
• ISSNs: 0956-7976

Abstract

Identifying brain biomarkers of disease risk is a growing priority in neuroscience. The ability to identify meaningful biomarkers is limited by measurement reliability; unreliable measures are unsuitable for predicting clinical outcomes. Measuring brain activity using task functional MRI (fMRI) is a major focus of biomarker development; however, the reliability of task fMRI has not been systematically evaluated. We present converging evidence demonstrating poor reliability of task-fMRI measures. First, a meta-analysis of 90 experiments ( N = 1, 008) revealed poor overall reliability—mean intraclass correlation coefficient (ICC) = .397. Second, the test-retest reliabilities of activity in a priori regions of interest across 11 common fMRI tasks collected by the Human Connectome Project ( N = 45) and the Dunedin Study ( N = 20) were poor (ICCs = .067–.485). Collectively, these findings demonstrate that common task-fMRI measures are not currently suitable for brain biomarker discovery or for individual-differences research. We review how this state of affairs came to be and highlight avenues for improving task-fMRI reliability.