JMIR Publications

ABSTRACT

Background:

Alzheimer's disease (AD) is a progressive neurodegenerative disorder posing challenges to patients, caregivers, and society. Accessible and accurate information is crucial for effective AD management.

Objective:

This study aimed to evaluate ChatGPT's ability to address AD management questions.

Methods:

Fourteen questions related to prevention, treatment, and care of AD were identified and posed to ChatGPT 3.5 and ChatGPT 4 in Chinese and English respectively. The four neurologists were asked to answer these 14 questions. We generated eight sets of responses and randomly coded them in answer sheets. Five neurologists and five patients' families rated responses using separate 5-point Likert scales.

Results:

Five neurologists and five family members of people with AD rated the 112 responses (GPT3.5: 28 responses, GPT4: 28 responses, Neurologists: 56 responses). Top five responses rated by neurologists had four ChatGPT responses and one physician's response. For the top five responses rated by patients' families, all but the third response were ChatGPT responses. The Neurologist-generated responses scored 3.9 ± 0.7, whereas the ChatGPT-generated responses scored 4.4 ± 0.6 (p<0.01). Language and model analyses revealed no significant differences in response quality between the GPT3.5 and GPT4 models. However, English responses outperformed Chinese responses in terms of comprehensibility and participant satisfaction. Chinese responses scored 4.4 ± 0.6, whereas English responses scored 4.5 ± 0.5 (p<0.01).

Conclusions:

ChatGPT is able to provide patient education materials on AD for patients, their families, nurses, caregivers and physicians. This capability can contribute to the effective healthcare management of Alzheimer's disease patients, leading to enhanced patient outcomes.