A validation study on the use of intra‐operative video recording as an objective assessment tool for core ENT surgery. (April 2014)

• Record Type: Journal Article
• Authors: Bowles, P.F.D.; Harries, M.; Young, P.; Das, P.; Saunders, N.; Fleming, J.C.
• Is Part Of: Clinical otolaryngology. Volume 39:Number 2(2014:Apr.)
• Page Start: 102
• Page End: 107
• Publication Date: 2014-04
• Subjects: Otolaryngology -- Periodicals; 617.51005
• DOI: 10.1111/coa.12240
• Languages: English
• ISSNs: 1749-4478

Abstract

<abstract abstract-type="main" id="coa12240-abs-0001"> <title> <x xml:space="preserve">Abstract</x> </title> <sec id="coa12240-sec-0001" sec-type="section"> <title>Objectives</title> <p>To investigate the reliability and validity of remote scoring a video assessment of a core ENT surgical procedure (myringotomy and grommet insertion) and its suitability as an objective tool for assessing the technical skills of ENT surgeons.</p> </sec> <sec id="coa12240-sec-0002" sec-type="section"> <title>Design</title> <p>Single‐blinded (raters) video assessment.</p> </sec> <sec id="coa12240-sec-0003" sec-type="section"> <title>Setting</title> <p>Tertiary Care University Hospital.</p> </sec> <sec id="coa12240-sec-0004" sec-type="section"> <title>Participants</title> <p>Consultant and trainee [Specialty Registrar (StR) and Core Trainee (CT)] ENT surgeons performing a total of 30 consecutive index procedures.</p> </sec> <sec id="coa12240-sec-0005" sec-type="section"> <title>Main outcome measures</title> <p>To determine the construct validity and the reliability of video assessment scoring of myringotomy and grommet insertion by two raters at ENT Consultant level with a subspecialty interest in Otology. To measure the performance (by rating) of participants compared to operative time.</p> </sec> <sec id="coa12240-sec-0006" sec-type="section"> <title>Results</title> <p>A strong correlation between scores by the two blinded raters was demonstrated (<italic>ρ</italic> = 0.748; <italic>P</italic> < 0.001). Median scores (/45) for each group were as follows: CT 25.5 (IQR 21.13–31.25), StR 33 (IQR 24.88–35) and Consultant 40 (IQR 35.38–42.63). Kruskal–Wallis test analysis showed statistically significant mean rank scores between the three different levels of experience (<italic>H</italic> = 12.77, <italic>P</italic> = 0.002). Multiple group comparisons indicated a significant difference between CT and Consultant groups (<italic>P</italic> < 0.001) and StR and Consultant groups (<italic>P</italic> = 0.007). Analysis of the time taken between the experience groups demonstrated a difference (<italic>H</italic> = 8.689, <italic>P</italic> = 0.013) although individual intergroup comparisons indicated this was only significant between CT and Consultant groups (<italic>P</italic> = 0.004). There was a significant negative correlation (<italic>ρ</italic> = −0.842; <italic>P</italic> < 0.001) between time taken for procedure and score achieved.</p> </sec> <sec id="coa12240-sec-0007" sec-type="section"> <title>Conclusions</title> <p>Video assessment of myringotomy and ventilation tube insertion may represent a valid, feasible tool for use in summative and formative assessments of trainee ENT surgeons. Remote scoring of assessment procedures minimises bias and enables blinding of raters. ENT is well positioned to benefit from video assessment due to the high number of surgical procedures within the specialty that are performed utilising digital technology.</p> </sec> </abstract>