Impact of virtual vs. in-person interviews among neurosurgery residency applicants. (July 2022)

• Record Type: Journal Article
• Authors: Alomari, Safwan; Lubelski, Daniel; Feghali, James; Brem, Henry; Witham, Timothy; Huang, Judy
• Is Part Of: Journal of clinical neuroscience. Volume 101(2022)
• Page Start: 63
• Page End: 66
• Publication Date: 2022-07
• Subjects: ICC intraclass correlation coefficient -- SD standard deviation -- I increase -- D decrease -- USMLE United Stated Medical Licensing Exam; Neurosurgery -- Residency application -- Interview -- Virtual -- Reliability; Brain -- Surgery -- Periodicals; Neurosciences -- Periodicals; Nervous system -- Surgery -- Periodicals; Brain -- surgery -- Periodicals; Neurosurgical Procedures -- Periodicals; Neurosciences -- Periodicals; Electronic journals; 616.8
• DOI: 10.1016/j.jocn.2022.05.005
• Languages: English
• ISSNs: 0967-5868

Abstract

Highlights: 214 neurosurgery residency applicants were interviewed between 2016 and 2021 at Johns Hopkins. Poor inter-rater agreement between faculty assessing applicants. Significant changes in variability of assessment scores for a given faculty each year. Virtual interviews and standardized questions may improve interview process. Abstract: Background: The interview is considered a key factor in selecting residents in various medical and surgical specialties. However, the reliability of the interview process in selecting neurosurgery training program applicants remains largely under-investigated. Objective: To investigate the reliability of the interview process for neurosurgery residency applicants and to evaluate the impact of virtual interviews on this process. Methods: We analyzed the records of neurosurgery residency applicant interviews at our institution between 2016 and 2021. An average of 20 neurosurgery faculty members (clinical and research) interviewed each applicant and graded them 1 (best) to 4 (worst). Intraclass correlation coefficient (ICC) and Levene's test were used to assess the inter-rater and intra-rater reliability, respectively. Results: 214 neurosurgery residency applicants were interviewed at a single institution between 2016 and 2021. The mean applicant rating each year ranged from 1.77 to 1.92. Inter-rater agreement was relatively poor in each year, (ICC < 0.5, P < 0.05). Among 60% of the raters, variability of scores significantly changed from year to year, (p < 0.05). When comparing the scores submitted during the virtual interview process (2021) with the scores submitted in the previous years (2016–2020), 2 interviewers (10%) had less variability using the virtual process. Conclusion: Our analysis found that the current interview process for neurosurgery residency applicants' selection suffers from poor inter- and intra-rater reliability. Virtual interviews may be part of a cost-effective strategy to improve the reliability of the interview process. Further validation is needed, as well as identification of novel strategies to maximize the reliability of the selection process.