Test-retest reliability on the Cambridge Neuropsychological Test Automated Battery: Comment on Karlsen et al. (2020). Issue 5 (3rd September 2022)

• Record Type: Journal Article
• Authors: Skirrow, Caroline; Cashdollar, Nathan; Granger, Kiri; Jennings, Sally; Baker, Elizabeth; Barnett, Jennifer; Cormack, Francesca
• Is Part Of: Applied neuropsychology. Volume 29:Issue 5(2022)
• Page Start: 889
• Page End: 892
• Publication Date: 2022-09-03
• Subjects: Cognition -- CANTAB -- measurement/statistics -- neuropsychology -- neuropsychology -- tests -- test-retest reliability; Neuropsychology -- Periodicals; Cognition disorders -- Periodicals; Brain -- Diseases -- Diagnosis -- Periodicals; Brain -- Imaging -- Periodicals; Brain -- Diseases -- Treatment -- Periodicals; Diagnostic Imaging -- Periodicals; Neuropsychological Tests -- Periodicals; Adult; 616.8
• DOI: 10.1080/23279095.2020.1860987
• Languages: English
• ISSNs: 2327-9095

Abstract

Abstract: Test-retest reliability is essential to the development and validation of psychometric tools. Here we respond to the article by Karlsen et al. (Applied Neuropsychology: Adult, 2020), reporting test-retest reliability on the Cambridge Neuropsychological Test Automated Battery (CANTAB), with results that are in keeping with prior research on CANTAB and the broader cognitive assessment literature. However, after adopting a high threshold for adequate test-retest reliability, the authors report inadequate reliability for many measures. In this commentary we provide examples of stable, trait-like constructs which we would expect to remain highly consistent across longer time periods, and contrast these with measures which show acute within-subject change in response to contextual or psychological factors. Measures characterized by greater true within-subject variability typically have lower test-retest reliability, requiring adequate powering in research examining group differences and longitudinal change. However, these measures remain sensitive to important clinical and functional outcomes. Setting arbitrarily elevated test-retest reliability thresholds for test adoption in cognitive research limits the pool of available tools and precludes the adoption of many well-established tests showing consistent contextual, diagnostic, and treatment sensitivity. Overall, test–retest reliability must be balanced with other theoretical and practical considerations in study design, including test relevance and sensitivity.