423 Trends in Utilization of Standardized Letters of Recommendation in the 2021-2022 Neurosurgery Application Process. (April 2023)

• Record Type: Journal Article
• Authors: Feldman, Michael; Ortiz, Alex; Roth, Steven G.; Dambrino, Robert J.; Yengo-Kahn, Aaron Michael; Chitale, Rohan V.; Chambless, Lola Blackwell
• Is Part Of: Neurosurgery. Volume 69(2023)Supplement 1
• Page Start: 83
• Page End: 83
• Publication Date: 2023-04
• Subjects: Nervous system -- Surgery -- Periodicals; 617.48005
• DOI: 10.1227/neu.0000000000002375_423
• Languages: English
• ISSNs: 0148-396X

Abstract

Abstract : INTRODUCTION: Standardized LORs (SLORs) were introduced during the 2020-2021 neurosurgery residency application cycle, but suffered from poor inter-rater reliability and significant grade inflation. Changes were made to the SLOR template and utilization patterns in response to these shortcomings. METHODS: This cross-sectional study analyzed all SLORs submitted to a single neurosurgery residency program over the 2021-2022 cycle. Data from seven competency domains and the overall rating were recorded and stratified by academic category of letter writer. Inter-rater reliability was evaluated using Krippendorff's Alpha. Frequency of letters written was evaluated using Kruskal-Wallis H Test. RESULTS: 90% of SLORs rated applicants among the Top 25%, but there was a significant decrease in the usage of the "Top 1%" and "Top 2-5%" ratings from the prior application cycle. Inter-rater reliability was poor across all competencies. Writers who completed one SLOR rated applicants higher and had a narrower range than those who completed multiple SLORs. CONCLUSIONS: Changes in the format and subsequent utilization patterns of SLORs have slightly decreased grade inflation; however, inter-rater reliability remains poor. The most wide-ranging evaluators submitted the highest number of SLORs, suggesting that future evaluation and usage of SLORs should emphasize letter-writer characteristics and numbers of SLORs written. Overall, SLORs have been well and broadly accepted with subtle improvements in the second year of utilization.