JMIR Publications

ABSTRACT

Background:

Hypertension, hyperlipidaemia, and hyperglycaemia, as widely recognized risk factors for cardiovascular disease (CVD) and other chronic conditions, are thus often included in chronic disease management and require long‑term pharmacological treatment and lifestyle modification. To enhance patient empowerment in managing these conditions, the Hospital Authority (HA) in Hong Kong has accelerated digital health adoption by enhancing its mobile application “HA Go” functions to particularly improve patients’ access to drug information. Artificial Intelligence (AI)-enabled tools, such as large language model (LLM)-powered chatbots, are increasingly integrated into healthcare settings to provide convenient access to drug information. However, reliance on fully AI-driven systems risks compromising accuracy and safety. Therefore, careful design, validation, and application of the chatbot are crucial to ensure information reliability.

Objective:

To develop a drug information chatbot that delivers verified drug information related to hypertension, hyperlipidaemia and hyperglycaemia and to conduct a preliminary evaluation of its performance and user experience.

Methods:

A multilingual (Cantonese, Mandarin, and English) and multimodal (text, image, and voice) WhatsApp chatbot was developed in collaboration with the Chief Pharmacist’s Office (CPO) of HA. LLM (GPT-4o) was used for semantic input analyses; outputs were generated from a HA pharmacist-validated knowledge database via retrieval-augmented generation (RAG) and confidence scoring. We conducted a 7-day single-arm usability trial on 13 participants (≥50 years old) with at least one of the medical conditions: hypertension, hyperlipidaemia, or hyperglycaemia. Pre- and post-trial questionnaires assessed users’ perceived knowledge, behavioural patterns, user experience, satisfaction, and collected qualitative feedback.

Results:

Before the trial, 69% of users reported having drug-related questions recently, primarily seeking information through healthcare providers (77%) and web searches (62%). Most users perceived adequate knowledge of the drug indications (77%) and dosage (69%), but fewer felt knowledgeable about side effects (46%). Post-trial assessment showed moderate usability ratings (mean 3-3.7 on a 5-point scale, with 5 denoting strongly agree); 77% of users intended future use, and 77% indicated willingness to recommend the chatbot to others. In-trial engagement with healthcare providers and web searches decreased to 31% and 38%, respectively. Users’ feedback highlighted the need to enhance interface usability and expand knowledge coverage.

Conclusions:

The LLM-powered chatbot delivered timely, evidence-based drug information while mitigating hallucination risks, showing strong potential as a supportive tool to assist patients in chronic disease management. Interface and content enhancements are actionable areas to improve patient experience. Clinical Trial: This study has been approved by the HKU/HA HKW Institutional Review Board (reference number: UW 23-518).