Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.
Who will be affected?
Readers: No access to all 28 journals. We recommend accessing our articles via PubMed Central
Authors: No access to the submission form or your user account.
Reviewers: No access to your user account. Please download manuscripts you are reviewing for offline reading before Wednesday, July 01, 2020 at 7:00 PM.
Editors: No access to your user account to assign reviewers or make decisions.
Copyeditors: No access to user account. Please download manuscripts you are copyediting before Wednesday, July 01, 2020 at 7:00 PM.
Quality and Accountability of ChatGPT in LMIC Healthcare: A Simulated Patient Study
Yafei Si;
Yuyi Yang;
Xi Wang;
Jiaqi Zu;
Xi Chen;
Xiaojing Fan;
Ruopeng An;
Sen Gong
ABSTRACT
Using simulated patients to mimic nine established non-communicable and infectious diseases over 27 trials, we assess ChatGPT’s effectiveness and reliability in diagnosing and treating common diseases in low- and middle-income countries. We find ChatGPT's performance varied within a single disease, despite a high level of accuracy in both diagnosis and medication prescription. Additionally, ChatGPT recommended a concerning level of unnecessary or harmful medications even with correct diagnoses. Finally, ChatGPT performed better in managing non-communicable diseases compared to infectious ones. These results highlight the need for cautious AI integration in healthcare systems to ensure quality and safety.
Citation
Please cite as:
Si Y, Yang Y, Wang X, Zu J, Chen X, Fan X, An R, Gong S
Quality and Accountability of ChatGPT in Health Care in Low- and Middle-Income Countries: Simulated Patient Study