Predictive validity of evidence-based practices in supported employment: a systematic review and meta-analysis. Issue 4 (12th December 2016) - Lockett, Helen; Waghorn, Geoffrey; Kydd, Rob; Chant, David
Abstract : Purpose: The purpose of this paper is to explore the predictive validity of two measures of fidelity to the individual placement and support (IPS) approach to supported employment. Design/methodology/approach: A systematic review and meta-analysis was conducted of IPS programs. In total, 30 studies provided information characterizing 69 cohorts and 8, 392 participants. Predictive validity was assessed by a precision and negative prediction analysis and by multivariate analysis of deviance. Findings: Fidelity scores on the IPS-15 scale of 60 or less accurately predicted poor outcomes, defined as 43 percent or less of participants commencing employment, in 100 percent of cohorts. Among cohorts with IPS-15 fidelity scores of 61-75, 63 percent attained good employment outcomes defined as 44 percent or more commencing employment. A similar pattern emerged from the precision analysis of the smaller sample of IPS-25 cohorts. Multivariate analysis of deviance for studies using the IPS-15 scale examined six cohort characteristics. Following adjustment for fidelity score, only fidelity score ( χ 2 =15.31, df=1, p <0.001) and author group ( χ 2 =35.01, df=17, p =0.01) representing an aspect of cohort heterogeneity, remained associated with commencing employment. Research limitations/implications: This study provides evidence of moderate, yet important, predictive validity of the IPS-15 scale across diverse international and research contexts. The smaller sample of IPS-25 studies limited the analysis that could be conducted. Practical implications: Program implementation leaders are encouraged to first focus on attaining good fidelity, then supplement fidelity monitoring with tracking the percentage of new clients who obtain a competitive job employment over a pre-defined period of time. Originality/value: The evidence indicates that good fidelity may be necessary but not sufficient for good competitive employment outcomes.