Comparison of Simulation‐based Resuscitation Performance Assessments With In‐training Evaluation Reports in Emergency Medicine Residents: A Canadian Multicenter Study. Issue 4 (19th September 2017)

• Record Type: Journal Article
• Authors: Hall, Andrew Koch; Damon Dagnone, J.; Moore, Sean; Woolfrey, Karen G. H.; Ross, John A.; McNeil, Gordon; Hagel, Carly; Davison, Colleen; Sebok‐Syer, Stefanie S.
• Editors: Santen, Sally
• Is Part Of: AEM education and training. Volume 1:Issue 4(2017)
• Page Start: 293
• Page End: 300
• Publication Date: 2017-09-19
• Subjects: Emergency medicine -- Study and teaching -- Periodicals; Emergency medicine -- Study and teaching -- United States -- Periodicals; Periodicals; 616.025
• DOI: 10.1002/aet2.10055
• Languages: English
• ISSNs: 2472-5390

Abstract

Abstract: Objective: Simulation stands to serve an important role in modern competency‐based programs of assessment in postgraduate medical education. Our objective was to compare the performance of individual emergency medicine (EM) residents in a simulation‐based resuscitation objective structured clinical examination (OSCE) using the Queen's Simulation Assessment Tool (QSAT), with portfolio assessment of clinical encounters using a modified in‐training evaluation report (ITER) to understand in greater detail the inferences that may be drawn from a simulation‐based OSCE assessment. Methods: A prospective observational study was employed to explore the use of a multicenter simulation‐based OSCE for evaluation of resuscitation competence. EM residents from five Canadian academic sites participated in the OSCE. Video‐recorded performances were scored by blinded raters using the scenario‐specific QSATs with domain‐specific anchored scores (primary assessment, diagnostic actions, therapeutic actions, communication) and a global assessment score (GAS). Residents' portfolios were evaluated using a modified ITER subdivided by CanMEDS roles (medical expert, communicator, collaborator, leader, health advocate, scholar, and professional) and a GAS. Correlational and regression analyses were performed comparing components of each of the assessment methods. Results: Portfolio review and ITER scoring was performed for 79 residents participating in the simulation‐based OSCE. There was a significant positive correlation between total OSCE and ITER scores (r = 0.341). The strongest correlations were found between ITER medical expert score and each of the OSCE GAS (r = 0.420), communication (r = 0.443), and therapeutic action (r = 0.484) domains. ITER medical expert was a significant predictor of OSCE total (p = 0.002). OSCE therapeutic action was a significant predictor of ITER total (p = 0.02) Conclusions: Simulation‐based resuscitation OSCEs and portfolio assessment captured by ITERs appear to measure differing aspects of competence, with weak to moderate correlation between those measures of conceptually similar constructs. In a program of competency‐based assessment of EM residents, a simulation‐based OSCE using the QSAT shows promise as a tool for assessing medical expert and communicator roles.