A novel workplace-based assessment for competency-based decisions and learner feedback. (2nd November 2018)

• Record Type: Journal Article
• Authors: Hicks, Patricia J.; Margolis, Melissa J.; Carraccio, Carol L.; Clauser, Brian E.; Donnelly, Kathleen; Fromme, H. Barrett; Gifford, Kimberly A.; Poynter, Sue E.; Schumacher, Daniel J.; Schwartz, Alan
• Is Part Of: Medical teacher. Volume 40:Number 11(2018)
• Page Start: 1143
• Page End: 1150
• Publication Date: 2018-11-02
• Subjects: Medical education -- Periodicals; 610.711
• DOI: 10.1080/0142159X.2018.1461204
• Languages: English
• ISSNs: 0142-159X

Abstract

Abstract: Background: Increased recognition of the importance of competency-based education and assessment has led to the need for practical and reliable methods to assess relevant skills in the workplace. Methods: A novel milestone-based workplace assessment system was implemented in 15 pediatrics residency programs. The system provided: (1) web-based multisource feedback (MSF) and structured clinical observation (SCO) instruments that could be completed on any computer or mobile device; and (2) monthly feedback reports that included competency-level scores and recommendations for improvement. Results: For the final instruments, an average of five MSF and 3.7 SCO assessment instruments were completed for each of 292 interns; instruments required an average of 4–8 min to complete. Generalizability coefficients >0.80 were attainable with six MSF observations. Users indicated that the new system added value to their existing assessment program; the need to complete the local assessments in addition to the new assessments was identified as a burden of the overall process. Conclusions: Outcomes – including high participation rates and high reliability compared to what has traditionally been found with workplace-based assessment – provide evidence for the validity of scores resulting from this novel competency-based assessment system. The development of this assessment model is generalizable to other specialties.