JMIR Publications

ABSTRACT

Background:

Generative artificial intelligence (AI) is transforming patient education in healthcare, particularly chatbots could give personalized and easy-to-understand medical information. This makes it crucial for the information to be both reliability and accuracy.

Objective:

The aim of this study is to compare the quality of responses from ChatGPT-4, ChatGPT-4o, and Google Gemini to common patient questions about spondylodiscitis and evaluated spine surgeons’ views on the reliability and role of AI in patient education.

Methods:

A pool of 50 relevant patient questions about spondylodiscitis was generated, and the ten most frequently recurring topics were selected and submitted to three different Large Language Models (LLMs). Seven experienced spine surgeons evaluated the responses using a scale from “excellent” to “unsatisfactory,” and further assessed comprehensiveness, clarity, empathy, and length. Statistical analysis was examined with the Mann-Whitney U-test.

Results:

Across all responses, 38.6% of responses were rated as excellent, 39.0% as satisfactory with minimal clarification needed, 16.7% as satisfactory with moderate clarification needed, and 5.7% as unsatisfactory. The most common reason for necessary clarification was insufficient information (40.7%), followed by language-related issues (15.9%) and overly detailed responses (12.8%). The question on potential complications of spondylodiscitis achieved the highest mean rating (3.4/5), whereas treatment- and prognosis-related questions received the lowest scores (2.7/5 and 2.9/5, respectively). Overall, participating spine surgeons expressed a generally positive attitude toward the use of AI in patient education.

Conclusions:

LLMs demonstrate considerable potential for supporting patient education in spondylodiscitis, receiving generally favorable feedback from clinical experts. Overall, LLMs appear suitable for providing patients with medical information; however, they should be integrated by clinicians and must not replace professional medical judgment. Future research should explore advanced, domain-specific models to further improve the quality of communication between clinicians and patients.