Exploring subjectivity in competency‐based assessment judgements of assessors. Issue 4 (9th November 2016)

• Record Type: Journal Article
• Authors: Bacon, Rachel; Holmes, Kay; Palermo, Claire
• Is Part Of: Nutrition & dietetics. Volume 74:Issue 4(2017)
• Page Start: 357
• Page End: 364
• Publication Date: 2016-11-09
• Subjects: assessment -- clinical nutrition and dietetics -- competency‐based education -- professional competence; Nutrition -- Periodicals; Dietetics -- Periodicals; 613.205
• DOI: 10.1111/1747-0080.12326
• Languages: English
• ISSNs: 1446-6368

Abstract

Abstract : Aim: The aim of this research was to measure variations in assessors' judgements of a student dietitian's performance and to explore the influence of group discussion on their judgements. Methods: The assessments of a student's performance, as observed from a video recording of an authentic nutrition consultation, were measured pre‐ and post‐group discussion by 26 experienced assessors using a mixed‐methods questionnaire. The instrument included a validated 7‐point visual analogue scale (VAS) rating (1 = novice; 7 = competent), a qualitative global description of performance and an assessor's confidence rating (1 = not at all confident; 10 = extremely confident). Scales were analysed descriptively and qualitative responses coded for key themes. Results: No agreement was found in assessors' rating in either the pretest (median = 4, range = 5) or post‐test (median = 4, range = 4); however, the discussion led 78% of participants (20/26) to change their VAS ratings (9/26) and/or confidence levels (16/26). Three themes emerged from the thematic analysis of the participants' global descriptions of performance: (i) discourse supports assessors to justify their judgements, identify assumptions and learn from the observations of others; (ii) discourse leads assessors to more holistic judgements; and (iii) multiple sources of evidence and student reflections are necessary for credible judgement. Conclusions: This research questions the notion that 'actual' performance can be objectively measured and, rather, considers assessments as 'interpretations'. This research calls for an integrated interpretivist student‐centred approach to competency‐based assessment.