A framework to identify ethical concerns with ML-guided care workflows: a case study of mortality prediction to guide advance care planning. (24th February 2023)

• Record Type: Journal Article
• Authors: Cagliero, Diana; Deuitch, Natalie; Shah, Nigam; Feudtner, Chris; Char, Danton
• Is Part Of: Journal of the American Medical Informatics Association. Volume 30:Number 5(2023)
• Page Start: 819
• Page End: 827
• Publication Date: 2023-02-24
• Subjects: machine learning -- clinical -- artificial intelligence -- ethics -- palliative care -- end-of-life care; Medical informatics -- Periodicals; Information Services -- Periodicals; Medical Informatics -- Periodicals; Médecine -- Informatique -- Périodiques; Informatica; Geneeskunde; Informatique médicale; Computer network resources; Electronic journals; 610.285
• DOI: 10.1093/jamia/ocad022
• Languages: English
• ISSNs: 1067-5027

Abstract

Abstract: Objective: Identifying ethical concerns with ML applications to healthcare (ML-HCA) before problems arise is now a stated goal of ML design oversight groups and regulatory agencies. Lack of accepted standard methodology for ethical analysis, however, presents challenges. In this case study, we evaluate use of a stakeholder "values-collision" approach to identify consequential ethical challenges associated with an ML-HCA for advanced care planning (ACP). Identification of ethical challenges could guide revision and improvement of the ML-HCA. Materials and Methods: We conducted semistructured interviews of the designers, clinician-users, affiliated administrators, and patients, and inductive qualitative analysis of transcribed interviews using modified grounded theory. Results: Seventeen stakeholders were interviewed. Five "values-collisions"—where stakeholders disagreed about decisions with ethical implications—were identified: (1) end-of-life workflow and how model output is introduced; (2) which stakeholders receive predictions; (3) benefit-harm trade-offs; (4) whether the ML design team has a fiduciary relationship to patients and clinicians; and, (5) how and if to protect early deployment research from external pressures, like news scrutiny, before research is completed. Discussion: From these findings, the ML design team prioritized: (1) alternative workflow implementation strategies; (2) clarification that prediction was only evaluated for ACP need, not other mortality-related ends; and (3) shielding research from scrutiny until endpoint driven studies were completed. Conclusion: In this case study, our ethical analysis of this ML-HCA for ACP was able to identify multiple sites of intrastakeholder disagreement that mark areas of ethical and value tension. These findings provided a useful initial ethical screening.