JMIR Publications

ABSTRACT

Background:

Patient education materials (PEMs) are essential for improving health literacy and clinical outcomes. Although large language models (LLMs) hold promise for optimizing content, their performance in generating new PEMs remains unclear.

Objective:

This study aimed to summarize the current evidence on LLMs in generating new PEMs, evaluate readability, quality, and usability in various health care setting, and guide future research and practice.

Methods:

This review followed PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. A comprehensive literature search was conducted in PubMed, Embase, Cochrane Library, Web of Science and Scopus from January 1, 2000, to January 1, 2026. Eligible studies were English-language original research conducted that evaluated new PEMs generated by LLMs in health care using information assessment tools. Two reviewers independently screened records and extracted data. we conducted a quantitative analysis of characteristics of the included literature. Furthermore, we classified the PEMs based on core educational tasks and clinical scenarios, and compared its readability, quality, and usability.

Results:

Thirty studies were included, including 11 LLMs and more than 30 diseases, most compared multiple LLMs and used a single prompt. PEMs were classified into five categories: Perioperative Patient Education (n=8), Chronic Disease Management Education (n=6), Specialty Disease Diagnostic and Therapeutic Education (n=9), Pharmacotherapy Safety and Adherence Education (n=3), and Public Health and Prevention Education (n=4). In terms of readability, only 3 records achieved a sixth-grade reading level. Specialty Disease Diagnostic and Therapeutic Education performed relatively well, while Chronic Disease Management Education had poor readability. The DeepSeek series demonstrated the most consistent performance. In terms of quality, most records were rated as good or higher. Specialty Disease Diagnostic and Therapeutic Education showed a polarized distribution, with both high and low quality. Pharmacotherapy Safety and Adherence Education and Public Health and Prevention Education were not rated as poor. The ChatGPT series performed relatively well. In terms of usability, most records performed well in understandability but poor in actionability. Perioperative Patient Education performed relatively balanced.

Conclusions:

LLMs offer a transformative opportunity for developing effective and high-quality patient education PEMs, but they currently fail to meet health literacy standards, particularly in readability and actionability. Future efforts must shift from performance evaluation to implementation strategy. Only through rigorous validation and equitable deployment can LLMs truly improve patient education.