Currently submitted to: JMIR Medical Education
Date Submitted: Aug 7, 2026
Open Peer Review Period: Aug 7, 2026 - Oct 2, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
The Additive Value of Training Evaluations for Predicting Early-Career Doctor Retention: A Real-World Evidence
ABSTRACT
Background:
The global shortage of physicians poses a significant threat to healthcare systems and health outcomes. While training and evaluations are identified as potential factors influencing physician retention, few studies have explored their predictive role.
Objective:
This study aims to investigate how clinical training evaluations can serve as predictors of early-career physicians’ retention, addressing a gap in the existing literature.
Methods:
This observational study analyzed data from a medical center in northern Taiwan, including 342 Clinical Competency Committee evaluation records of 128 postgraduate year (PGY) physicians. The evaluation records were collected at three six-month intervals for each trainee. Evaluations included multiple competency assessments, such as Direct Observation of Procedural Skills (DOPS), Mini-Clinical Evaluation Exercise, Case-based Discussion, and 360-degree feedback from peers, patients, and clinical teachers. A Random Forest predictive model was developed to identify key factors associated with physician retention, with a training cohort (70%) and a testing cohort (30%) used for internal validation.
Results:
The analysis revealed that DOPS and multi-source feedback, particularly from peers and patients, were the most important predictors of physicians' retention at the institution. Other factors, such as demographic characteristics and academic performance, showed minimal impact on retention outcomes. The Random Forest model demonstrated high accuracy, with an Area Under the Curve of 93.2% (95% CI: 88.2% to 98.1%), correctly predicting retention in 93.75% of cases in which physicians left and 76.32% of cases in which they stayed (p<0.001).
Conclusions:
This study underscores the potential of clinical training evaluations as early indicators of physician retention. Procedural competence and feedback from peers and patients emerged as key factors influencing retention. These findings imply that training evaluations can serve as predictive tools for physician retention. By identifying at-risk physicians early, healthcare institutions can implement targeted strategies to improve retention and ensure a sustainable workforce. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.