What should the standard be for passing and mastery on the Critical Thinking about Health Test? A consensus study. Issue 2 (24th February 2023)

• Record Type: Journal Article
• Authors: Nsangi, Allen; Aranza, Diana; Asimwe, Roger; Munaabi-Babigumira, Susan Kyomuhendo; Nantongo, Judith; Nordheim, Lena Victoria; Ochieng, Robert; Oyuga, Cyril; Uwimana, Innocent; Dahlgren, Astrid; Oxman, Andrew
• Is Part Of: BMJ open. Volume 13:Issue 2(2023)
• Publication Date: 2023-02-24
• Subjects: medical education & training -- community child health -- public health -- education & training (see medical education & training) -- health services administration & management; Medicine -- Research -- Periodicals; 610.72
• DOI: 10.1136/bmjopen-2022-066890
• Languages: English
• ISSNs: 2044-6055

Abstract

Abstract : Objective: Most health literacy measures rely on subjective self-assessment. The Critical Thinking about Health Test is an objective measure that includes two multiple-choice questions (MCQs) for each of the nine Informed Health Choices Key Concepts included in the educational resources for secondary schools. The objective of this study was to determine cut-off scores for passing (the border between having and not having a basic understanding and the ability to apply the nine concepts) and mastery (the border between having mastered and not having mastered them). Design: Using a combination of two widely used methods: Angoff's and Nedelsky's, a panel judged the likelihood that an individual on the border of passing and another on the border of having mastered the concepts would answer each MCQ correctly. The cut-off scores were determined by summing up the probability of answering each MCQ correctly. Their independent assessments were summarised and discussed. A nominal group technique was used to reach a consensus. Setting: The study was conducted in secondary schools in East Africa. Participants: The panel included eight individuals with 5 or more years' experience in the following areas: evaluation of critical thinking interventions, curriculum development, teaching of lower secondary school and evidence-informed decision-making. Results: The panel agreed that for a passing score, students had to answer 9 of the 18 questions and for a mastery score, 14 out of 18 questions correctly. Conclusion: There was wide variation in the judgements made by individual panel members for many of the questions, but they quickly reached a consensus on the cut-off scores after discussions.