Predictive validity of evidence-based practices in supported employment: a systematic review and meta-analysis. Issue 4 (12th December 2016)

• Record Type: Journal Article
• Authors: Lockett, Helen; Waghorn, Geoffrey; Kydd, Rob; Chant, David
• Is Part Of: Mental health review. Volume 21:Issue 4(2016)
• Page Start: 261
• Page End: 281
• Publication Date: 2016-12-12
• Subjects: Fidelity -- Severe mental illness -- Supported employment -- Predictive validity -- Evidence-based practices; Mental health services -- Great Britain -- Periodicals; Mental health -- Periodicals; Mental Health Services -- Great Britain -- Periodicals; 362.20941
• DOI: 10.1108/MHRJ-12-2015-0040
• Languages: English
• ISSNs: 1361-9322

Abstract

Abstract : Purpose: The purpose of this paper is to explore the predictive validity of two measures of fidelity to the individual placement and support (IPS) approach to supported employment. Design/methodology/approach: A systematic review and meta-analysis was conducted of IPS programs. In total, 30 studies provided information characterizing 69 cohorts and 8, 392 participants. Predictive validity was assessed by a precision and negative prediction analysis and by multivariate analysis of deviance. Findings: Fidelity scores on the IPS-15 scale of 60 or less accurately predicted poor outcomes, defined as 43 percent or less of participants commencing employment, in 100 percent of cohorts. Among cohorts with IPS-15 fidelity scores of 61-75, 63 percent attained good employment outcomes defined as 44 percent or more commencing employment. A similar pattern emerged from the precision analysis of the smaller sample of IPS-25 cohorts. Multivariate analysis of deviance for studies using the IPS-15 scale examined six cohort characteristics. Following adjustment for fidelity score, only fidelity score ( χ 2 =15.31, df=1, p <0.001) and author group ( χ 2 =35.01, df=17, p =0.01) representing an aspect of cohort heterogeneity, remained associated with commencing employment. Research limitations/implications: This study provides evidence of moderate, yet important, predictive validity of the IPS-15 scale across diverse international and research contexts. The smaller sample of IPS-25 studies limited the analysis that could be conducted. Practical implications: Program implementation leaders are encouraged to first focus on attaining good fidelity, then supplement fidelity monitoring with tracking the percentage of new clients who obtain a competitive job employment over a pre-defined period of time. Originality/value: The evidence indicates that good fidelity may be necessary but not sufficient for good competitive employment outcomes.