Simulation-Based Assessment of Critical Care "Front-Line" Providers*. Issue 6 (June 2018)

• Record Type: Journal Article
• Authors: Boyle, Walter A.; Murray, David J.; Beyatte, Mary Beth; Knittel, Justin G.; Kerby, Paul W.; Woodhouse, Julie; Boulet, John R.
• Is Part Of: Critical care medicine. Volume 46:Issue 6(2018)
• Publication Date: 2018-06
• Subjects: clinical competence -- critical care -- educational measurement -- intensive care units -- nurse practitioners -- patient simulation; Critical care medicine -- Periodicals; Soins intensifs -- Périodiques; 616.028
• DOI: 10.1097/CCM.0000000000003073
• Languages: English
• ISSNs: 0090-3493

Abstract

Abstract : Objectives: Develop a standardized simulation method to assess clinical skills of ICU providers. Design: Simulation assessment. Setting: Simulation laboratory. Subjects: Residents, Critical Care Medicine fellows, acute care nurse practitioner students. Interventions: Performance scoring in scenarios from multiple Critical Care Medicine competency domains. Measurements and Main Results: Three-hundred eighty-four performances by 48 participants were scored using checklists (% correct) and holistic "global" ratings (1 [unprepared] to 9 [expert]). One-hundred eighty were scored by two raters. Mean checklist and global scores (± SD) ranged from 65.0% (± 16.3%) to 84.5% (± 17.3%) and 4.7 (± 1.4) to 7.2 (± 1.2). Checklist and global scores for Critical Care Medicine fellows and senior acute care nurse practitioner students (Experienced group, n = 26) were significantly higher than those for the Novice acute care nurse practitioner students (Novice group, n = 14) (75.6% ± 15.6% vs 68.8% ± 21.0% and 6.1 ± 1.6 vs 5.4 ± 1.5, respectively; p < 0.05). Residents (Intermediate group, n = 8) scored between the two (75.4% ± 18.3% and 5.7 ± 1.7). 38.5% of the Experienced group scored in the top quartile for mean global score, compared with 12.5% of the Intermediate and 7.1% of the Novice groups. Conversely, 50% of the Novice group scored in the lower quartile (< 5.3), compared with 37.5% of the Intermediate and 11.5% of the Experienced groups. Psychometric analyses yielded discrimination values greater than 0.3 for most scenarios and reliability for the eight-scenario assessments of 0.51 and 0.60, with interrater reliability of 0.71 and 0.75, for checklist and global scoring, respectively. Conclusions: The simulation assessments yielded reasonably reliable measures of Critical Care Medicine decision-making skills. Despite a wide range of performance, those with more ICU training and experience performed better, providing evidence to support the validity of the scores. Simulation-based assessments may ultimately prove useful to determine readiness to assume decision-making roles in the ICU. Abstract : Supplemental Digital Content is available in the text.