Feasibility of an artificially intelligent vaccines chatbot in Pakistan: A mixed methods evaluation. (20th October 2021)

• Record Type: Journal Article
• Authors: Munir, M; Miraj, F; Raza, H; Hussain, OA; Khan, AA; Siddiqi, DA; Khan, AJ; Habib, A; Chandir, S
• Is Part Of: European journal of public health. Volume 31(2021)Supplement 3
• Publication Date: 2021-10-20
• Subjects: Epidemiology -- Europe -- Periodicals; Public health -- Europe -- Periodicals; 362.109405
• DOI: 10.1093/eurpub/ckab164.507
• Languages: English
• ISSNs: 1101-1262

Abstract

Abstract: Background: Despite free vaccination services, gaps in information access impede immunization uptake in low and middle-income countries (LMICs), including Pakistan. We developed Bablibot (Babybot), a local-language, text-based chatbot to connect caregivers with immunization-related information in real-time, and evaluated its feasibility and acceptability in resolving immunization-related queries among Karachi's low-income communities. Methods: We developed Bablibot using Natural Language Processing (NLP), Machine Learning, and Human in the Loop features. We conducted a sequential mixed-methods pilot in Karachi where caregivers were recruited when visiting selected immunization centers for any childhood vaccine except Measles 2 (last vaccine visit) or were informed through targeted text messages. We assessed Bablibot's feasibility and acceptability through user-engagement metrics and thematic analysis of qualitative interviews conducted with 20 pilot participants. Results: Between March 9, 2020 and April 15, 2021, Bablibot accrued 677 users and participated in 874 conversations. Of 2, 202 pilot-enrolled caregivers, 210 (10%) interacted with the bot; other 467 users were not pilot participants. Most queries were about due dates (25%; 221/874), delaying vaccination (17%; 146/874), or side-effect management (16%;137/874). The conversations now constitute a valuable training database for future local-language bots. Over 90% (277/307) of responses to text-based exit surveys indicated user satisfaction. Qualitative analysis showed caregivers appreciate Bablibot's convenience, reliability, and responsiveness. Conclusions: Our results demonstrate the feasibility and acceptability of local-language NLP chatbots in providing immunization information. Text-based chatbots can minimize workload on helpline operators, in addition to quickly resolving queries that otherwise lead to delay or default. Bablibot's conversation data provides learning infrastructure for future health-related bots. Key messages: An AI-based NLP chatbot is a feasible and acceptable intervention for providing immunization-related information to large number of caregivers 24/7 in limited resource setting. By serving as a remote, low-cost and confidential bi-directional channel, chatbots can address gaps in information access, especially for women, and have long term impact on immunization uptake.