Resident Surgical Skills Web-Based Evaluation: A Comparison of 2 Assessment Tools. (6th March 2019)

• Record Type: Journal Article
• Authors: Van Heest, Ann E.; Agel, Julie; Ames, S. Elizabeth; Asghar, Ferhan A.; Harrast, John J.; Marsh, J. Lawrence; Patt, Joshua C.; Sterling, Robert S.; Peabody, Terrance D.
• Is Part Of: Journal of bone and joint surgery. Volume 101:Number 5(2019)
• Publication Date: 2019-03-06
• Subjects: Bones -- Surgery -- Periodicals; Joints -- Surgery -- Periodicals; Orthopedics -- Periodicals; Orthopedics; General Surgery; Bone Diseases; Joint Diseases; Bones -- Surgery; Joints -- Surgery; Orthopedics; Bot (anatomie); Gewrichten; Chirurgie (geneeskunde); Periodicals; Electronic journals; Periodicals; 617.47005
• DOI: 10.2106/JBJS.17.01512
• Languages: English
• ISSNs: 0021-9355

Abstract

Abstract : Background: Evaluation of surgical skill competency is necessary as graduate medical education moves toward a competency-based curriculum. This study by the American Board of Orthopaedic Surgery (ABOS) and the Council of Orthopaedic Residency Directors (CORD) compares 2 web-based evaluation tools that assess the level of autonomy that is demonstrated by residents during surgical procedures in the operating room as measured by faculty. Methods: Two hundred and ninety-four residents from 16 orthopaedic surgery residency programs were evaluated by 370 faculty using 2 web-based evaluation tools in a crossover design in which residents requested faculty review of their surgical skills before starting a case. One thousand, one hundred and fifty Ottawa Surgical Competency Operating Room Evaluation (O-Score) assessments, which included a 9-question evaluation of 8 steps of the surgical procedure, were compared with 1, 186 P-score evaluations, which included a single-question summative evaluation. Twenty-five different surgical procedures were evaluated. Results: There were no significant differences in rates of resident requests or faculty completion of the 2 scores. The most common surgical procedures that were assessed were total knee arthroplasty (n = 254, 11%), carpal tunnel release (n = 191, 8%), open reduction and internal fixation (ORIF) of stable hip fractures (n = 170, 7%), ORIF of simple ankle fractures (n = 169, 7%), and total hip arthroplasty (n = 166, 7%). Both instruments disclosed significant differences in competency among entry, intermediate, and advanced-level residents. The findings support the construct validity of the evaluation method. The survey results indicated that >70% of the faculty were confident that use of either the P-score or the O-score allowed them to distinguish a resident who can perform the surgery independently from one who needs additional training. Conclusions: This research has led to the modification of the O-score and the P-score into a combined OP-score instrument. The ABOS envisions that the OP-score instrument can be used with an expanded number of surgical procedures as a required element of residency training in the near future. Clinical Relevance: This study allows the profession of orthopaedic surgery education to take a leadership role in the measurement of competence for surgical skills for orthopaedic surgeons in residency training, an important clinically relevant topic to the practice of orthopaedic surgery.