The usefulness of work-based assessments in higher surgical training: A systematic review. (October 2021)

• Record Type: Journal Article
• Authors: Aryal, Kamal; Hamed, Mazin; Currow, Chelise
• Is Part Of: International journal of surgery. Volume 94(2021)
• Publication Date: 2021-10
• Subjects: Work-based assessments -- Procedure based assessment -- Clinical evaluation exercise -- Surgical training -- Formative assessment -- Case based discussion -- Direct observation of procedural skills; Surgery -- Periodicals; Surgical Procedures, Operative -- Periodicals; 617.005
• DOI: 10.1016/j.ijsu.2021.106127
• Languages: English
• ISSNs: 1743-9191

Abstract

Abstract: Background: Work-based assessments including procedure based assessments, case based discussions, clinical evaluation exercises and direct observation of procedural skills are used in Higher Surgical Training Program. This systematic review aims to investigate the trainer and trainee's perception of the usefulness of workplace based assessments in Higher Surgical Training Programme. Materials and methods: Embase, MEDLINE and PubMed databases were searched for relevant studies published up to Jan 15, 2021. The following search terms were used: procedure based assessments, case based discussions, clinical evaluation exercises, direct observation of procedural skills, Higher Surgical Training Program (with and without their abbreviations), surgical training, formative assessment. Usefulness was analysed according to van der Vleuten's utility formula, which is the product of educational impact, validity, reliability, acceptability, cost-effectiveness and feasibility. Results: 23 studies were included; Six on procedure based assessments, two on case based discussions, one on mini clinical evaluation exercises, three on direct observation of procedural skills and eleven on multiple methods. Overall, procedure based assessments had positive Kirkpatrick level 1 or 2 impact were valid, reliable and acceptable. Mini clinical evaluation exercises showed positive Kirkpatrick level 1 satisfaction in trainees and trainers and were feasible. Case based discussions were shown to have both positive and negative Kirkpatrick levels 1 and 2 impact and were rated valid and reliable. Direct observation of procedural skills were valid with favourable Kirkpatrick level 1 impact with more usage and modification of the forms. Multiple methods as used in the intercollegiate surgical curriculum programme portfolio was found to be less encouraging in achieving positive higher level of educational impact. The barriers included lack of time, lack of faculty development and lack of engagement. Conclusion: It is important that work-based assessments are used properly, so that trainees get the maximum benefit from them. None of the studies included in our systematic review demonstrated an impact on behaviour or society (Kirkpatrick level 3 and 4), towards which future research should be directed to. Highlights: Workplace based assessments (WBAs) are used in higher surgical training. This is up to date summary of the literature evidence on their usefulness. They were found to be useful when used individually or study conditions. Multiple methods as used in the ISCP portfolio was found to be less encouraging. The barriers on using are lack of time, faculty development and engagement.