471 Assessing the Accuracy and Bias of Digital Symptom Checkers with Myocardial Infarction Patients. (19th August 2022)

• Record Type: Journal Article
• Authors: Wallace, W; Chan, C; Chidambaram, S; Hanna, L; Iqbal, F; Acharya, A; Normahani, P; Ashrafian, H; Markar, S; Sounderajah, V; Darzi, A
• Is Part Of: British journal of surgery. Volume 109(2022)Supplement 6
• Publication Date: 2022-08-19
• Subjects: Surgery -- Periodicals; 617.005
• DOI: 10.1093/bjs/znac269.105
• Languages: English
• ISSNs: 0007-1323

Abstract

Abstract: Aim: The accuracy and safety of symptom checkers in diagnosing and triaging patients is of concern; especially those with life-threatening conditions. The study's aims were to: 1. assess the accuracy of symptom checkers in diagnosing and triaging myocardial infarctions (MI) and, 2. determine whether differences in gender or presentation type exist. Method: This prospective diagnostic accuracy study assessed 8 symptom checkers using 100 MI patients of various presentations: typical or atypical. The ability of a symptom checker in providing MI as the first diagnosis (D1) and the first 3 (D3) diagnoses were diagnostic accuracy measures. Triage advice was deemed correct if the symptom checker recommended seeking emergency treatment. Results: Symptom checkers correctly diagnosed 48.0±31.4% of cases with MI first. D3 accuracy was 72.6±20.2%. Mean triage accuracy was 82.6±12.6%. 24.0±16.2% of atypical cases had a correct primary diagnosis. D3 accuracy for atypical MI was 43.8±20.6%, significantly lower than that of typical MI (p<0.01). Atypical case triage accuracy was 52.7±20.0%, significantly lower than typical cases (84.2±14.7%, p<0.01). 10.0% of the atypical female cases were diagnosed correctly with MI as the first diagnosis. Female atypical cases had significantly lower accuracy than typical female cases for all accuracy measures (p<0.01). Conclusions: Symptom checkers generally provide low accuracy for diagnosing MI. Approximately 20% of cases were under-triaged. Results varied between symptom checkers: patients who presented with atypical symptoms tended to be under-diagnosed and under-triaged, especially those who were female. This demonstrated potential gender bias and therefore raises questions regarding symptom checker regulation and safety.