Currently submitted to: JMIR Medical Education
Date Submitted: Oct 20, 2025
Open Peer Review Period: Oct 21, 2025 - Dec 16, 2025
(closed for review but you can still tweet)
NOTE: This is an unreviewed Preprint
Warning: This is a unreviewed preprint (What is a preprint?). Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn (a note "no longer under consideration" will appear above).
Peer review me: Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period (in this case, a "Peer Review Me" button to sign up as reviewer is displayed above). All preprints currently open for review are listed here. Outside of the formal open peer-review period we encourage you to tweet about the preprint.
Citation: Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author).
Final version: If our system detects a final peer-reviewed "version of record" (VoR) published in any journal, a link to that VoR will appear below. Readers are then encourage to cite the VoR instead of this preprint.
Settings: If you are the author, you can login and change the preprint display settings, but the preprint URL/DOI is supposed to be stable and citable, so it should not be removed once posted.
Submit: To post your own preprint, simply submit to any JMIR journal, and choose the appropriate settings to expose your submitted version as preprint.
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Can AI Improve Medical School Admissions? A Reliability Analysis of AI-Generated Multiple Mini Interview (MMI) Stations.
ABSTRACT
Background:
Multiple Mini Interviews (MMIs) are widely used in medical school admissions to assess applicants’ non-academic attributes in a structured and reliable manner. However, the development of high-quality MMI stations is resource-intensive and dependent on expert input.
Objective:
This study explores the utility of artificial intelligence (AI) in the generation MMI stations for the Direct and Graduate Entry Medicine Program admissions process for domestic applicants at Monash Medical School.
Methods:
A total of 56 MMI stations from the 2025 admissions cycle were evaluated, including 17 AI-generated and 39 traditionally developed stations, administered across 824 domestic applicants for a total of 4,897 applicant-station interactions. We assessed station quality through both reliability (using Cronbach’s alpha to examine internal consistency) and discrimination capability (using standard deviation and range of scores) at the station level.
Results:
AI-generated stations exhibited slightly higher reliability (α = 0.8181) compared to existing stations (α = 0.8081), though this difference was not statistically significant. Both AI-generated and traditionally developed stations demonstrated variable discrimination capability, with some stations from each development method showing excellent combinations of high reliability and strong discriminatory power, while others exhibited ceiling effects that limited their discriminatory power. Importantly, a greater proportion of AI-generated stations achieved excellent reliability (α > 0.85), and a lower proportion demonstrated poor reliability (α < 0.75), compared to traditional stations, suggesting that AI-generated development may enhance the consistency and quality of MMI stations.
Conclusions:
Our findings presented here highlight the utility of AI as a useful tool for MMI station generation, offering a scalable approach that may reduce the resource burden on faculty while maintaining or enhancing psychometric quality for applicants. Ongoing quality assurance and evaluation remain essential to ensure fairness and validity across the admissions process.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.