G642(P) Introducing a minimum accepted competency (MAC) exam for commencing supervised paediatric practice. (May 2019)

• Record Type: Journal Article
• Authors: McCrossan, P; McCallion, N
• Is Part Of: Archives of disease in childhood. Volume 104:(2019)Supplement 2
• Page Start: A260
• Page End: A260
• Publication Date: 2019-05
• Subjects: Children -- Diseases -- Periodicals; Infants -- Diseases -- Periodicals; 618.920005
• DOI: 10.1136/archdischild-2019-rcpch.621
• Languages: English
• ISSNs: 0003-9888

Abstract

Abstract : Aims: To determine if undergraduate students and paediatric trainee doctors level of knowledge meets a non-faculty clinician-determined minimum accepted competency (MAC). Methods: A 30-item multiple-choice (MCQ) paper (MAC exam) was created, formed of questions proposed by practising non-academic consultant paediatricians, which are deemed as 'must know ' for paediatric trainees prior to commencing clinical work. A 'passing score' was determined using the Angoff technique by the paediatric faculty. The paper was given to undergraduate students following their formal paediatric teaching and also paediatric senior house officers (SHO's). Student's performance on the MAC was compared with their performance on their official university examination. Test item analysis and psychometrics were also performed. Results: The passing score was determined at 13/30 (41.2%). 366 undergraduate students sat the exam. Their mean score was 45.9% (s.d 9.9%, range 23%–73%). 240/366 (65.4%) of students achieved the passing score whereas 99% of these students passed their official university exam. However, there was a positive correlation between students result in the MAC exam and results from their official university exams (spearman R=0.44, p<0.01). 58 paediatric SHO's sat the exam. Their mean score was 64.2% (s.d 11.8%, range 40%–80%) which was significantly better than undergraduate students (p<0.01). After ranking test item difficulty, there was a positive correlation in responses between the students and the SHO's (spearman R=0.73 (p<0.01). We identified a pattern of consistently poorly answered questions in order to highlight areas of knowledge deficit. Conclusion: This is a novel approach to paediatric assessment, enabling clinicians as opposed to academics to design the content. The MAC exam is more difficult than expected and both students and SHO's found the same questions challenging. Students results were significantly worse on the MAC compared with their official university exam. We need to explore further if this is due to a lack of knowledge (requiring curriculum change) or unreasonable expectations from clinicians. The MAC exam showed reproducible results but other psychometric properties are not yet strong enough for it to be considered as a high stakes assessment tool in its current format.