Accepted for/Published in: JMIR Research Protocols
Date Submitted: Jul 2, 2024
Open Peer Review Period: Jul 2, 2024 - Aug 27, 2024
Date Accepted: Dec 27, 2024
(closed for review but you can still tweet)
ChatGPT4 Performance on German CME - friend or foe (trick or treat)?
ABSTRACT
Background:
The increasing development and spread of artificial and assistive intelligence is opening up new areas of application not only in applied medicine but also in related fields such as continuing medical education (CME), which is part of the mandatory training program for medical doctors in Germany.
Objective:
The aim of our study was to determine whether medical laypersons are able to successfully conduct training courses specifically for physicians with the help of a large language model such as ChatGPT4.
Methods:
We plan to conduct a randomized controlled trial in which high school students use ChatGPT4 to complete special training courses for doctors in the fields of internal medicine, surgery, gynecology, pediatrics, neurology and anesthesiology. Thus, the test is set up with three arms: a) input of full-text and CME questions in ChatGPT4, b) input of CME questions only and c) a solution approach using a keyword search function in the full text. By means of randomization, the participants were evenly distributed among the three arms. The trial was approved by the Ethics Committee of Witten/Herdecke University (No. S-108/2024, date of approval 15 May 2024) and registered on the Open Science Framework (https://doi.org/10.17605/OSF. IO/MZNUF) in advance.
Results:
not yet applicable.
Conclusions:
Using this approach, we wanted to test further possible applications of artificial intelligence in the postgraduate medical education setting and obtain results for practical use. Depending on the results, the potential influence of LLMs such as Chat-GPT4 on CME will be discussed, e.g., as part of a SWOT analysis (strengths, weaknesses, opportunities, threats). Clinical Trial: Open Science Framework (https://doi.org/10.17605/OSF. IO/MZNUF)
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.