Decision-making bias in assessment: The effect of aggregating objective information and anecdote. (October 2013)

• Record Type: Journal Article
• Authors: Tweed, Mike J.; Thompson-Fawcett, Mark; Wilkinson, Tim J.
• Is Part Of: Medical teacher. Volume 35:Number 10(2013)
• Page Start: 832
• Page End: 837
• Publication Date: 2013-10
• Subjects: Medical education -- Periodicals; 610.711
• DOI: 10.3109/0142159X.2013.803062
• Languages: English
• ISSNs: 0142-159X

Abstract

<abstract> <title> <x xml:space="preserve">Abstract</x> </title> <p> <bold>Introduction</bold>: Assessment decisions increasingly rely on synthesis of information from a variety of sources. It is known that aggregation of information to make decisions is open to a number of biases. The aim of this research was to investigate bias, accuracy and confidence of assessment decision making.</p> <p> <bold>Methods</bold>: The participants were consultation skills assessors. A model for incremental information was developed with participants being shown results from purposefully selected, but authentic, data from the University's final summative 10-station Objective Structured Clinical Examination (OSCE). After each piece of information, participants gave a pass-fail decision and their confidence in that choice. Following the information from 10 OSCE stations the participants were given a discordant fictional anecdote and again participants gave a pass-fail decision and their confidence.</p> <p> <bold>Results</bold>: When there is overwhelming evidence to support a pass or fail, participants were not as confident as the data would support. Participants were less confident to make a fail decision than a pass. Despite considerable evidence from multiple results some participants altered decisions based on isolated contradictory information from an anecdote.</p> <p> <bold>Discussion</bold>: These findings are significant in understanding decision-making. Given equivalent levels of evidence, decision makers are less confident to fail than pass and less robust information can undermine more robust information.</p> </abstract>