P057: Performance of a national simulation-based resuscitation OSCE for emergency medicine trainees. (2nd June 2016)

• Record Type: Journal Article
• Authors: Hagel, C.; Hall, A.K.; Klinger, D.; McNeil, G.; Dagnone, D.
• Is Part Of: CJEM. Volume 18(2016:Jan.)Supplement 1
• Page Start: S97
• Page End: S98
• Publication Date: 2016-06-02
• Subjects: simulation, -- objective structured clinical examination (OSCE), -- competency; Emergency Treatment -- Periodicals; Emergency Medicine -- Periodicals; Emergency medical services -- Canada -- Periodicals; Medical emergencies -- Canada -- Periodicals; Emergency medical services; Medical emergencies; Canada; Periodicals; 616.02505
• DOI: 10.1017/cem.2016.233
• Languages: English
• ISSNs: 1481-8035

Abstract

Abstract : Introduction: The use of high-fidelity simulation is emerging as an effective method for competency-based assessment in postgraduate medical education. We have previously reported the development of the Queen's Simulation Assessment Tool (QSAT), for use in simulation-based Objective Structured Clinical Examinations (OSCEs) for Emergency Medicine (EM) trainees. We aimed to demonstrate the feasibility and present an argument for the validity of a simulation-based OSCE utilizing the QSAT with EM residents from multiple Canadian training sites.Methods: EM post-graduate trainees (PGY 2-5) from 9 Canadian EM training programs participated in an 8-station simulation-based resuscitation OSCE at Queen's University in Kingston, ON. Each station was scored by a single trained rater from a group of 9 expert Canadian EM physicians. Raters utilized a station-specific QSAT and provided an Entrustment Score. A post-examination questionnaire was administered to the trainees to quantify perceived realism, comfort and educational impact. Statistical analyses included analysis of variance to measure the discriminatory capabilities and a generalizability study to examine the sources of variability in the scores.Results: EM postgraduate trainees (N=36) participated in the study. Discriminatory validity was strong, with senior trainees (PGY4-5) outperforming junior trainees (PGY2-3) in 6 of 8 scenarios and in aggregated QSAT and Entrustment Scores across all 8 stations (p<0.01). Generalizability studies found the largest sources of random variability was due to the trainee by station interaction and the error term, with a G coefficient of 0.84. Resident trainees reported reasonable comfort being assessed in the simulation environment (3.6/5), indicated significant perceived realism (4.1/5), and found the OSCE valuable to their learning (4.8/5).Conclusion: Overall, this study demonstrates that a large-scale simulation-based EM resuscitation OSCE is feasible, and an argument has been presented for the validity of such an examination. The incorporation of simulation or a simulation-based OSCE in the national certification process in EM may help to satisfy the increased demand for competency-based assessment required by the Royal College of Physicians & Surgeons of Canada's Competency by Design transition.