Assessment of the Non-Technical Skills for Surgeons (NOTSS) framework in the USA. Issue 9 (23rd April 2020)

• Record Type: Journal Article
• Authors: Pradarelli, J C; Gupta, A; Lipsitz, S; Blair, P Gabler; Sachdeva, A K; Smink, D S; Yule, S
• Is Part Of: British journal of surgery. Volume 107:Issue 9(2020)
• Page Start: 1137
• Page End: 1144
• Publication Date: 2020-04-23
• Subjects: Surgery -- Periodicals; 617.005
• DOI: 10.1002/bjs.11607
• Languages: English
• ISSNs: 0007-1323

Abstract

Abstract: Background: Surgeons' non-technical skills are important for patient safety. The Non-Technical Skills for Surgeons assessment tool was developed in the UK and recently adapted to the US surgical context (NOTSS-US). The aim of this study was to evaluate the reliability and distribution of non-technical skill ratings given by attending (consultant) surgeons who underwent brief online training. Methods: Attending surgeons across six specialties at a large US academic medical centre underwent a 10-min online training, then rated 60-s standardized videos of simulated operations. Intraclass correlation coefficient (ICC), and mean(s.d.) values for NOTSS-US ratings were determined for each non-technical skill category (score range 1–5, where 1 indicates poor, 3 average and 5 excellent) and for total NOTSS-US score (range 4–20; sum of 4 category scores). Outcomes were adjusted for rater characteristics including sex, specialty and clinical experience. Results: A total of 8889 ratings were submitted by 81 surgeon raters on 30 simulated intraoperative videos. The mean(s.d.) total NOTSS-US score for all videos was 9·5(4·8) of 20. The within-video ICC for total NOTSS-US score was 0·64 (95 per cent c.i. 0·57 to 0·70). For individual non-technical skill categories, the ICC was highest for social skills (communication/teamwork: 0·63, 95 per cent c.i. 0·56 to 0·71; leadership: 0·64, 0·55 to 0·72) and lowest for cognitive skills (situation awareness: 0·54, 0·45 to 0·62; decision-making: 0·50, 0·41 to 0·59). Women gave higher total NOTSS-US scores than men (adjusted mean difference 0·93, 95 per cent c.i. 0·44 to 1·43; P = 0·001). Conclusion: After brief online training, the inter-rater reliability of the NOTSS-US assessment tool achieved moderate strength among trained surgeons rating simulated intraoperative videos. Graphical Abstract: The Non-Technical Skills for Surgeons (NOTSS) assessment tool was recently adapted to the US surgical context. This prospective cohort study evaluated a scalable method of training and assessing surgeons' non-technical skills by characterizing the reliability and distribution of NOTSS-US ratings given by attending US surgeons who underwent this training. Moderate inter-rater reliability