Currently submitted to: JMIR Medical Education
Date Submitted: Mar 30, 2026
Open Peer Review Period: Apr 2, 2026 - May 28, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Estimating the Prevalence of Generative AI Use in Medical School Application Essays: A Cross-Sectional Study
ABSTRACT
Background:
Generative artificial intelligence (AI) tools became widely available to the public in November 2022. The extent to which these tools are being used by medical school applicants during the admissions process is unknown.
Objective:
We aimed to estimate the extent of generative AI use in cohorts of applicants spanning the rollout of these tools.
Methods:
We retrospectively analyzed 6,000 essays submitted to a U.S. medical school in 2021–2022 (baseline, before wide availability of AI) and in 2023–2024 (test year) to estimate the prevalence of AI use and its relation to other application data. We used GPTZero, a commercially available detection tool, to generate a metric (P_human) for the likelihood that each essay was human-generated, ranging from 0 (entirely AI) to 1 (entirely human).
Results:
Fully human-generated negative controls demonstrated a median P_human of 0.93, while AI-generated positive controls demonstrated a median P_human of 0.01. Personal Comments essays submitted in the 2023–2024 cycle had a median P_human of 0.77 (95% confidence interval 0.76–0.78), versus 0.83 (95% CI 0.82–0.85) during the 2021–2022 cycle. Approximately 12.3 and 2.7% of essays were evaluated as having P_human < 0.5 in the test and baseline year, respectively. Secondary essays demonstrated lower P_human than AMCAS Personal Comments essays, suggesting more AI use. In multivariate analysis, younger age and higher GPA were significantly associated with lower P_human. No differences were observed in gender, MCAT score, undergraduate major, or socioeconomic status. P_human was not predictive of admissions outcomes in uni- or multivariate analyses.
Conclusions:
An AI detection algorithm identified significantly increased use of generative AI in 2023–2024 medical school admission applications, as compared to the 2021–2022 baseline. AI use did not appear to confer an admissions advantage. While these results provide information about the applicant pool as a whole, AI detection is imperfect. We do not recommend deploying AI detection on individual applications in live admissions cycles. Clinical Trial: Trial Registration: None
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.