Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Accepted for/Published in: JMIR Research Protocols

Date Submitted: Jul 2, 2024
Open Peer Review Period: Jul 2, 2024 - Aug 27, 2024
Date Accepted: Dec 27, 2024
(closed for review but you can still tweet)

The final, peer-reviewed published version of this preprint can be found here:

ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial

Burisch C, Bellary A, Breuckmann F, Ehlers J, Thal SC, Sellmann T, Gödde D

ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial

JMIR Res Protoc 2025;14:e63887

DOI: 10.2196/63887

PMID: 39913914

PMCID: 11843049

Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

ChatGPT4 Performance on German CME - friend or foe (trick or treat)?

  • Christian Burisch; 
  • Abhav Bellary; 
  • Frank Breuckmann; 
  • Jan Ehlers; 
  • Serge C Thal; 
  • Timur Sellmann; 
  • Daniel Gödde

ABSTRACT

Background:

The increasing development and spread of artificial and assistive intelligence is opening up new areas of application not only in applied medicine but also in related fields such as continuing medical education (CME), which is part of the mandatory training program for medical doctors in Germany.

Objective:

The aim of our study was to determine whether medical laypersons are able to successfully conduct training courses specifically for physicians with the help of a large language model such as ChatGPT4.

Methods:

We plan to conduct a randomized controlled trial in which high school students use ChatGPT4 to complete special training courses for doctors in the fields of internal medicine, surgery, gynecology, pediatrics, neurology and anesthesiology. Thus, the test is set up with three arms: a) input of full-text and CME questions in ChatGPT4, b) input of CME questions only and c) a solution approach using a keyword search function in the full text. By means of randomization, the participants were evenly distributed among the three arms. The trial was approved by the Ethics Committee of Witten/Herdecke University (No. S-108/2024, date of approval 15 May 2024) and registered on the Open Science Framework (https://doi.org/10.17605/OSF. IO/MZNUF) in advance.

Results:

not yet applicable.

Conclusions:

Using this approach, we wanted to test further possible applications of artificial intelligence in the postgraduate medical education setting and obtain results for practical use. Depending on the results, the potential influence of LLMs such as Chat-GPT4 on CME will be discussed, e.g., as part of a SWOT analysis (strengths, weaknesses, opportunities, threats). Clinical Trial: Open Science Framework (https://doi.org/10.17605/OSF. IO/MZNUF)


 Citation

Please cite as:

Burisch C, Bellary A, Breuckmann F, Ehlers J, Thal SC, Sellmann T, Gödde D

ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial

JMIR Res Protoc 2025;14:e63887

DOI: 10.2196/63887

PMID: 39913914

PMCID: 11843049

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.