JMIR Publications

ABSTRACT

Background:

Diabetes requires continuous management to prevent complications. Many patients rely on online search engines and mobile health apps for information, but these tools often lead to information overload, complex navigation, and lack of personalization. Generative AI chatbots have emerged as promising alternatives by providing accessible and tailored health recommendations, addressing each patient's unique needs more effectively.

Objective:

This study explores the potential of generative AI chatbots in diabetes management. It investigates the types of questions diabetes patients frequently ask, evaluates user experience, and compares chatbot interactions with online health information-seeking behaviors and physician consultations. Additionally, the study gathers expert insights by having family medicine specialists review chatbot conversation logs and interact with the chatbot to assess its utility, limitations, and future applications in diabetes care.

Methods:

A two-part evaluation study was conducted. In Part 1, 24 patients with diabetes participated in structured sessions, followed by surveys and interviews to gather user insights. In Part 2, four family medicine specialists reviewed conversation logs to evaluate response accuracy and provided feedback on the chatbot’s potential role in diabetes care. Both parts of the study utilized DTalksBot, a GPT-4-based chatbot enhanced with Retrieval-Augmented Generation (RAG), to ensure reliable and contextually relevant responses.

Results:

Patients valued the chatbot’s real-time and personalized guidance, emphasizing its ability to simplify health information-seeking and reduce cognitive burden. Family medicine specialists acknowledged its potential to support routine care by addressing lifestyle-related questions, reducing provider workload, and tracking patient concerns for follow-up discussions. However, they noted the need for more tailored responses and integration with real-time patient data, such as glucose monitoring or treatment history, to better align with personalized care standards.

Conclusions:

Generative AI chatbots exhibited promise as supplementary tools for diabetes management, providing reliable, personalized, and accessible health information. Unlike traditional online searches that often lead to information overload, these chatbots mitigate this issue by offering verified guidance, supporting informed decision-making, and enhancing patient engagement in self-management. However, their full potential requires further developments, such as dynamic personalization, integration with verified clinical data, and alignment with healthcare workflows to ensure safety and effectiveness. Future enhancements could establish generative AI chatbots as comprehensive, patient-centered tools for managing chronic diseases.