Clinical-chatbot AHP evaluation based on "quality in use" of ISO/IEC 25010. (February 2023)

• Record Type: Journal Article
• Authors: Barletta, Vita Santa; Caivano, Danilo; Colizzi, Lucio; Dimauro, Giovanni; Piattini, Mario
• Is Part Of: International journal of medical informatics. Volume 170(2023)
• Publication Date: 2023-02
• Subjects: Medical-chatbot quality -- Clinical pathway -- AHP -- ISO/IEC 25010; Medical informatics -- Periodicals; Information science -- Periodicals; Computers -- Periodicals; Medical technology -- Periodicals; Medical Informatics -- Periodicals; Technology, Medical -- Periodicals; Computers; Information science; Medical informatics; Medical technology; Electronic journals; Periodicals; Electronic journals; 610.285
• DOI: 10.1016/j.ijmedinf.2022.104951
• Languages: English
• ISSNs: 1386-5056

Abstract

Highlights: We propose a set of measures for each characteristic of ISO/IEC 25010. A quantitative method is proposed for making homogeneous the AHP pairwise weights. As a case study, a comparison of two versions of a chatbot was performed. Abstract: Background : Conversational agents are currently a valid alternative to humans in first-level interviews with users who need information, even in-depth, about services or products. In application domains such as health care, this technology can become pervasive only if the perceived "quality in use" is appropriate. How to measure chatbot quality is an open question. The international standard ISO/IEC 25010 proposes a set of characteristics (effectiveness, efficiency, satisfaction, freedom from risk, and context coverage) to be considered when the "quality in use" of a software system has to be measured. Basic procedure : This study proposes a clinical chatbot comparison method based on quality. The proposed approach is based on Analytic Hierarchy Process methodology (AHP). Findings : Our contribution is twofold. First, we propose a set of measures for each characteristic of ISO/IEC 25010 according to three classes of functionality: providing information, providing prescriptions and process management. Moreover a quantitative method is proposed for making homogeneous the pairwise weights when the AHP is used for the "quality-in-use" comparison. As a case study, a comparison of two versions of a chatbot was performed. Conclusions : The results show that the proposed approach provides an effective reference base for performing quality comparisons of medical chatbots compliant with the ISO/IEC 25010 standard.