JMIR Publications

ABSTRACT

Background:

Large language models (LLMs) are increasingly used to generate case-based materials for health professions education (HPE). Although previous reviews have examined broad applications in HPE or focused on LLM-generated clinical cases, none has comprehensively synthesized how LLMs are used to generate and evaluate a broad range of case-based materials, or the associated findings and challenges.

Objective:

To synthesize current evidence on the use of LLMs to generate case-based materials in HPE, focusing on generation approaches, evaluation methods, key findings, and reported challenges.

Methods:

This scoping review followed the Joanna Briggs Institute methodology and was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). Seven databases (PubMed, Embase, Scopus, Web of Science, CINAHL, ERIC, and IEEE Xplore) were searched on May 10, 2026, for studies published from January 1, 2017. Eligible studies were peer-reviewed original research articles, published in English, reporting the use of generative LLMs to create text-based case-based materials for HPE. Data were synthesized using descriptive statistics and narrative synthesis.

Results:

Seventy-two studies were included. Most were published between 2024 and 2026 (67/72, 93%) and focused on medical education (59/72, 82%), with undergraduate students the most frequently studied population. Case-based multiple-choice questions (28/72, 39%) and clinical cases or vignettes (22/72, 30%) were the most frequently generated materials. GPT-4 family models were the most commonly used (40/72, 55%), and generation predominantly relied on relatively simple prompting approaches, whereas more advanced strategies, such as few-shot prompting, prompt chaining, or use of external knowledge sources, were rarely adopted. Evaluation primarily relied on expert review, typically involving small panels, while student assessment (25/72, 35%) and psychometric analyses were less common. Overall, LLM-generated materials were frequently judged to be clinically plausible, structurally clear, and reasonably aligned with educational objectives, particularly after refinement. However, consistent limitations included factual inaccuracies, construction flaws, superficial clinical realism, pedagogical misalignment, suboptimal psychometric performance, and bias. No LLM consistently outperformed others, with performance varying by educational task.

Conclusions:

This review provides a comprehensive synthesis of the literature on LLM-generated case-based materials in HPE. Unlike previous reviews examining broad educational applications or focusing on clinical cases, it integrates evidence across a broad range of case-based materials while synthesizing generation methods, evaluation approaches, reported findings, and challenges. The evidence suggests that LLMs can support the development of case-based materials but still require expert oversight because of recurring concerns regarding clinical accuracy, educational quality, psychometric performance, clinical realism, and bias. By identifying methodological and evidence gaps, this review provides educators, curriculum developers, and researchers with a structured overview of this rapidly evolving field and may inform the development of more robust generation workflows, more standardized evaluation approaches, and the responsible use of LLM-generated case-based materials in HPE.