Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.
Who will be affected?
Readers: No access to all 28 journals. We recommend accessing our articles via PubMed Central
Authors: No access to the submission form or your user account.
Reviewers: No access to your user account. Please download manuscripts you are reviewing for offline reading before Wednesday, July 01, 2020 at 7:00 PM.
Editors: No access to your user account to assign reviewers or make decisions.
Copyeditors: No access to user account. Please download manuscripts you are copyediting before Wednesday, July 01, 2020 at 7:00 PM.
Benchmarking State-of-the-Art Large Language Models for Migraine Patient Education: A Comparison of Performances on the Responses to Common Queries.
Linger Li;
Pengfei Li;
Kun Wang;
Liang Zhang;
Hongqin Zhao;
Hongwei Ji
ABSTRACT
Migraine, a frequent and highly disabling disorder, necessitates enhanced education of individuals with migraine to mitigate this global burden. The rapidly evolving field of large language models (LLMs) presents a promising avenue for assisting in migraine patient education. This study aims to assess the potential of LLMs in this context by evaluating the accuracy of responses from five leading LLMs, including OpenAI's ChatGPT 3.5 and 4.0, Google Bard, Meta Llama2, and Anthropic Claude2, in addressing 30 commonly asked migraine-related queries. We found that LLMs demonstrated varied levels of accuracy. ChatGPT-4.0 provided 96.7% appropriate responses, while other chatbots provided 83.3% to 90% appropriate responses. This study underscores the potential of LLMs, notably ChatGPT-4.0, demonstrated to accurately address common migraine-related queries. Such findings could advance AI-assisted education for individuals with migraine, providing insights for a holistic approach to migraine management.
Citation
Please cite as:
Li L, Li P, Wang K, Zhang L, Zhao H, Ji H
Benchmarking State-of-the-Art Large Language Models for Migraine Patient Education: Performance Comparison of Responses to Common Queries