Use of role plays to assess therapist competency and its association with client outcomes in psychological interventions: A scoping review and competency research agenda. (July 2020)

• Record Type: Journal Article
• Authors: Ottman, Katherine E.; Kohrt, Brandon A.; Pedersen, Gloria A.; Schafer, Alison
• Is Part Of: Behaviour research and therapy. Volume 130(2020)
• Publication Date: 2020-07
• Subjects: Psychological treatments -- Training -- Competence -- Developing countries -- Common mental disorders -- Paraprofessionals; Cognitive therapy -- Periodicals; Psychotherapy -- Periodicals; 616.891
• DOI: 10.1016/j.brat.2019.103531
• Languages: English
• ISSNs: 0005-7967

Abstract

Abstract: A major challenge in scaling-up psychological interventions worldwide is how to evaluate competency among new workforces engaged in psychological services. One approach to measuring competency is through standardized role plays. Role plays have the benefits of standardization and reliance on observed behavior rather than written knowledge. However, role plays are also resource intensive and dependent upon inter-rater reliability. We undertook a two-part scoping review to describe how competency is conceptualized in studies evaluating the relationship of competency with client outcomes. We focused on use of role plays including achieving inter-rater reliability and the association with client outcomes. First, we identified 4 reviews encompassing 61 studies evaluating the association of competency with client outcomes. Second, we identified 39 competency evaluation tools, of which 21 were used in comparisons with client outcomes. Inter-rater reliability (intraclass correlation coefficient) was reported for 15 tools and ranged from 0.53 to 0.96 (mean ICC = 0.77). However, we found that none of the outcome comparison studies measured competency with standardized role plays. Instead, studies typically used therapy quality (i.e., session ratings with actual clients) as a proxy for competency. This reveals a gap in the evidence base for competency and its role in predicting client outcomes. We therefore propose a competency research agenda to develop an evidence-base for objective, standardized role plays to measure competency and its association with client outcomes. Open science registration #: https://osf.io/nqhu7/ Graphical abstract: Chart of Findings for Evaluated Therapist Competency and Client Outcomes.HIC, high-income countries; LMIC, low- and middle-income countries. Image 1 Highlights: There are mixed findings regarding the relationship between therapist competency and client outcomes. Therapy quality is typically used as a proxy for therapist competency. No studies to date have used standardized role plays to assess competency's relationship to client outcomes. A research agenda is described for measuring competency and client outcomes in accord with expert recommendations.