Accepted for/Published in: JMIR Medical Education
Date Submitted: Apr 5, 2025
Date Accepted: Jan 20, 2026
Effectiveness of Informed AI Utilization on Clinical Competence of General Practitioners and Internists: A Pre–Post Intervention Study
ABSTRACT
Background:
Artificial intelligence (AI) shows promise in clinical diagnosis, treatment support, and healthcare efficiency. However, its adoption in real-world practice remains limited due to insufficient clinical validation and unclear impact on practitioners' competence. Addressing these gaps is essential for effective, confident, and ethical AI integration into modern healthcare settings.
Objective:
To determine the effectiveness of informed AI use, following tailored AI training course, on the clinical competence and attitudes of general practitioners (GPs) and internists toward AI clinical applications.
Methods:
A pre- and post-intervention study was conducted with 326 physicians from 39 countries. Participants completed a baseline test of clinical decision-making covering skills of diagnosis, treatment planning, and patient counseling, attended a 1.5-hour online training on effective AI use, and then took a similar post-training test with ChatGPT available as an assistant. Test performance and time per question were compared pre- and post-training. Participants also rated AI accuracy, efficiency, perceived need for structured AI training, and their willingness to use AI in practice before and after the course.
Results:
Average test scores improved from 57% to 78% (p < 0.001), with larger gains among GPs and younger physicians. All skill domains improved significantly, while time per correct response increased slightly with AI assistance. By the end of the intervention, physicians viewed AI more favorably-reporting increased confidence in its accuracy and time efficiency, greater appreciation for the need for structured training, and increased confidence and willingness to integrate AI into patient care.
Conclusions:
Informed use of AI, based on tailored training, substantially enhanced GPs’ and internists’ clinical decision-making skills and confidence in using AI. These findings support integrating AI training into medical education and continuing professional development to improve clinical performance and promote competent use of AI in clinical practice.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.