JMIR Publications

ABSTRACT

Background:

The growing impact of GenAI on medical education has given rise to many reviews discussing AI literacy, trustworthiness, potential uses and institutional or faculty benefits. However, there is surprisingly little focus on its impact on student learning, with students generally being positioned as technology users rather than learners. This approach risks prioritising educational technology innovation over actual pedagogical impact. Therefore, there is a need to place students back at the centre of GenAI-educational interventions.

Objective:

This scoping review examines the impact of GenAI-enabled educational interventions on medical undergraduate students.

Methods:

We followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. An electronic search was conducted between 22–24 September 2025 across Ovid (Medline, Embase), PubMed, Web of Science and Scopus with no language or date restrictions and excluding reviews, commentaries, letters, editorials, preprints, conference papers, and study protocols. Each record was independently screened by two reviewers who thoroughly discussed and agreed upon its final inclusion in the analysis. We used a modified version of the Kirkpatrick model both to guide inclusion and as an analytical framework for the final dataset.

Results:

27 studies were included. Most were published in 2025 (21/27) and were conducted in twelve different countries. The educational interventions described included simulated patient experience, knowledge or decision support and content generation. The majority of studies fell under Kirkpatrick level 1, focussing on initial user reactions and providing minimal insight into learning gains. The studies that fell under Kirkpatrick level 2 examined learning outcomes in terms of knowledge gain, skills acquisition and learning attitude, however, the strength of evidence supporting learning gains varied widely. No studies were classified as Kirkpatrick level 3. Three studies demonstrated explicit theoretical grounding and therefore were classified as Kirkpatrick level 4 in our modified model. Further, during the screening process, we found numerous studies on use cases without student involvement (117) or which focussed on GenAI performance without considering its impact on student learning (114). Although these studies were not included in the final analysis, they revealed the extent to which student learning has been neglected when investigating GenAI-enabled educational interventions.

Conclusions:

Our review highlights the volume of studies that prioritise tool performance over learning processes, as well as a trend towards fragmentation of educational design elements in research evaluating GenAI-enabled educational interventions. Therefore, and based on our findings, we developed the Technology-Integrated Educational Design (TIED) roadmap which reimagines the Kirkpatrick model as a tool to guide student-centred, theory-driven and iterative design of technology-enabled educational interventions. We urge researchers to direct attention away from evaluating learner reactions, learning outcomes and tool performance, towards deeper understanding of the processes and mechanisms underlying how students learn with technology.