JMIR Publications

ABSTRACT

Background:

Rising healthcare demands driven by ageing populations and increasing chronic disease burden have strained healthcare systems globally. Hospital at home models, supported by mobile health and remote patient monitoring technologies, offer a promising approach to deliver hospital-grade care in patients’ homes.

Objective:

Building on the original Dr Buddy platform deployed within Singapore General Hospital’s SGH@Home program, this study describes the design, implementation, and evaluation of three new features integrated into Dr Buddy 2.0.

Methods:

Three new features were developed and integrated: a Large Language Model-based Disease-Specific Question and Answering System, a Clinical Decision Tree-based Automated Check-In, and a Large Language Model-assisted Pre-Video Consultation module. Evaluation of the Disease-Specific Question and Answering System was conducted through automated metrics (answer relevancy and faithfulness) and clinical assessment. Usability and acceptability of the Automated Check-In Module and Large Language Model-based Pre-Video Consultation were evaluated through an adapted Chatbot Usability Questionnaire and mHealth App Usability Questionnaire items and open-ended questions.

Results:

Automated evaluation of the Disease-Specific Question and Answering System demonstrated strong performance across both open-source and closed-source models, achieving high relevance scores (> 0.8) and near-perfect faithfulness scores (> 0.98). Clinician evaluation further supported the system’s performance, with an overall rating of 8.28 out of 10. A total of 19 patients enrolled into SGH@Home program between October 2025 and March 2026 agreed to use Dr Buddy 2.0 features in addition to their daily reporting of vital signs, and 13 healthcare providers interacted with Dr Buddy 2.0 in their care provision. Patient usability scores for the Clinician Decision Tree-based Automated Check-In Module and Large Language Model-based Pre-Video Consultation were high, with patients particularly valuing ease of use, timely reminders, and confidence in transmission of data. Healthcare providers reported moderate usability scores, with the highest ratings for the chatbot’s acceptability as a care delivery tool. Qualitative feedback highlighted the chatbot’s accessibility and reliability, while identifying areas for improvement, including error handling, scope of responses, and support for older users.

Conclusions:

Dr Buddy 2.0 demonstrates that it is a clinically grounded tool for home-based care. Future development should prioritize accessibility for older adults and include a formal assessment of operational impact on clinical workload.