Predicting Performance of First-Year Residents: Correlations Between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study. (March 2019)

• Record Type: Journal Article
• Authors: Marcus-Blank, Brittany; Dahlke, Jeffrey A.; Braman, Jonathan P.; Borman-Shoap, Emily; Tiryaki, Ezgi; Chipman, Jeffrey; Andrews, John S.; Sackett, Paul R.; Cullen, Michael J.
• Is Part Of: Academic medicine. Volume 94:Number 3(2019)
• Publication Date: 2019-03
• Subjects: Medical education -- Periodicals; Medical policy -- Periodicals; Medical personnel -- Periodicals; Periodicals; 610.711
• DOI: 10.1097/ACM.0000000000002429
• Languages: English
• ISSNs: 1040-2446

Abstract

Abstract : Purpose: To determine whether scores on structured interview (SI) questions designed to measure noncognitive competencies in physicians (1) predict subsequent first-year resident performance on Accreditation Council for Graduate Medical Education (ACGME) milestones and (2) add incremental validity over United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge scores in predicting performance. Method: The authors developed 18 behavioral description questions to measure key noncognitive competencies (e.g., teamwork). In 2013–2015, 14 programs (13 residency, 1 fellowship) from 6 institutions used subsets of these questions in their selection processes. The authors conducted analyses to determine the validity of SI and USMLE scores in predicting first-year resident milestone performance in the ACGME's core competency domains and overall. Results: SI scores predicted midyear and year-end overall performance ( r = 0.18 and 0.19, respectively, P < .05) and year-end performance on patient care, interpersonal and communication skills, and professionalism competencies ( r = 0.23, r = 0.22, and r = 0.20, respectively, P < .05). SI scores contributed incremental validity over USMLE scores in predicting year-end performance on patient care ( ΔR = 0.05), interpersonal and communication skills ( ΔR = 0.09), and professionalism ( ΔR = 0.09; all P < .05). USMLE scores contributed incremental validity over SI scores in predicting year-end performance overall and on patient care and medical knowledge. Conclusions: SI scores predict first-year resident year-end performance in the interpersonal and communication skills, patient care, and professionalism competency domains. Future research should investigate whether SIs predict a range of clinically relevant outcomes.