JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI), particularly large language models and other generative systems, is increasingly being used in medical education for assessment-related tasks. However, the assessment-specific literature remains difficult to interpret because existing overviews have largely focused on AI in medical education more broadly rather than on learner assessment and feedback.

Objective:

This study aimed to map the literature on AI for assessment and feedback in medical education by examining temporal publication trends, leading contributors and collaboration patterns, assessment functions and settings, learner stages, and reporting of key domains related to validity, reliability, fairness, integrity, transparency, human oversight, implementation, and governance.

Methods:

We conducted a bibliometric analysis combined with a thematic evidence map. Web of Science Core Collection, Scopus, and PubMed were searched from January 1, 2015, to April 8, 2026. Assessment was defined as any AI-supported activity used to generate, score, interpret, or provide feedback on learner performance or assessment content in a medical education context. Records were identified through topic-based retrieval, targeted journal-based supplementary retrieval, and citation-based backfilling. After multistage deduplication, document selection was conducted primarily at the title and abstract level, with supplementary web-assisted and PDF-assisted ambiguity resolution when needed.

Results:

The search identified 14,968 records, of which 8378 remained after multistage deduplication. Following document selection, 435 records comprised the final screened analysis cohort. Annual output increased sharply after 2022, and 399 of 435 records (91.7%) were published in the post-ChatGPT period. The leading journals by record count were BMC Medical Education (33/435, 7.6%) and JMIR Medical Education (24/435, 5.5%), and the United States was the largest contributing country and principal collaboration hub. Generative AI (310/435, 71.3%) and large language models (301/435, 69.2%) were the dominant AI categories. Summative assessment was the most common assessment function (168/435, 38.6%), followed by feedback (93/435, 21.4%) and question generation (59/435, 13.6%). Board-style examinations (151/435, 34.7%) and written examinations (88/435, 20.2%) were the most common settings, and undergraduate medical education was the most represented learner stage (172/435, 39.5%). Reporting was uneven across evaluation and governance domains. Implementation was reported in 212/435 records (48.7%), reliability in 107/435 (24.6%), and validity in 84/435 (19.3%), whereas fairness (45/435, 10.3%), human oversight (46/435, 10.6%), transparency (19/435, 4.4%), and integrity (11/435, 2.5%) were infrequently reported.

Conclusions:

The literature on AI for assessment and feedback in medical education has expanded rapidly, particularly in the post-ChatGPT period, but remains concentrated in generative-AI applications, summative and examination-oriented contexts, and undergraduate medical education. More authentic assessment settings, broader stages of training, and explicit reporting of fairness, integrity, transparency, and human oversight remain underdeveloped. Future research should move beyond examination benchmarking toward more educationally authentic and implementation-ready uses of AI in assessment.