Trust in AI: why we should be designing for APPROPRIATE reliance. (2nd November 2021)

• Record Type: Journal Article
• Authors: Benda, Natalie C; Novak, Laurie L; Reale, Carrie; Ancker, Jessica S
• Is Part Of: Journal of the American Medical Informatics Association. Volume 29:Number 1(2022)
• Page Start: 207
• Page End: 212
• Publication Date: 2021-11-02
• Subjects: artificial intelligence -- trust -- algorithms -- machine learning; Medical informatics -- Periodicals; Information Services -- Periodicals; Medical Informatics -- Periodicals; Médecine -- Informatique -- Périodiques; Informatica; Geneeskunde; Informatique médicale; Computer network resources; Electronic journals; 610.285
• DOI: 10.1093/jamia/ocab238
• Languages: English
• ISSNs: 1067-5027

Abstract

Abstract: Use of artificial intelligence in healthcare, such as machine learning-based predictive algorithms, holds promise for advancing outcomes, but few systems are used in routine clinical practice. Trust has been cited as an important challenge to meaningful use of artificial intelligence in clinical practice. Artificial intelligence systems often involve automating cognitively challenging tasks. Therefore, previous literature on trust in automation may hold important lessons for artificial intelligence applications in healthcare. In this perspective, we argue that informatics should take lessons from literature on trust in automation such that the goal should be to foster appropriate trust in artificial intelligence based on the purpose of the tool, its process for making recommendations, and its performance in the given context. We adapt a conceptual model to support this argument and present recommendations for future work.