Accepted for/Published in: JMIR AI
Date Submitted: May 9, 2024
Open Peer Review Period: May 9, 2024 - Jul 4, 2024
Date Accepted: Nov 8, 2024
(closed for review but you can still tweet)
GPT-4 as a Clinical Decision Support Tool in Ischemic Stroke Management: Evaluation study
ABSTRACT
Background:
Cerebrovascular diseases are the second most common cause of death worldwide and one of the major causes of disability burden. Advancements in artificial intelligence (AI) have the potential to revolutionize healthcare delivery, particularly in critical decision-making scenarios such as ischemic stroke management.
Objective:
Here, we evaluated the effectiveness of GPT-4 in providing clinical support for emergency room neurologists comparing its recommendations with expert opinions and real-world outcomes.
Methods:
A cohort of 100 patients with acute stroke symptoms was retrospectively reviewed. Data used for decision-making included patients’ history, clinical evaluation, imaging study results, and other relevant details. Each case was independently presented to GPT-4, which provided a scaled recommendation (1-7) regarding the appropriateness of treatment, the use of tissue plasminogen activator, and the need for endovascular thrombectomy. Additionally, GPT-4 estimated the 90-day mortality probability for each patient and elucidated its reasoning for each recommendation. The recommendations were then compared with a stroke specialist and actual treatment decision.
Results:
Agreement of GPT-4’s recommendations with the expert opinion yielded an AUC of 0.85 [95% CI: 0.77-0.93], and with real-world treatment decisions, an AUC of 0.80 [0.69-0.91]. Mortality prediction, GPT-4 accurately identified 10 out of 13 within its top 25 high-risk predictions (AUC = 0.89 [95% CI: 0.8077-0.9739]; HR: 6.98 [95% CI: 2.88-16.9]), surpassing supervised machine-learning models.
Conclusions:
This study demonstrates the potential of GPT-4 as a viable clinical decision-support tool in the management of acute stroke. Its ability to provide explainable recommendations without requiring structured data input aligns well with the routine workflows of treating physicians. Future studies should focus on prospective validations and exploring the integration of such AI tools into clinical practice.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.