JMIR Publications

ABSTRACT

Background:

Patients with comorbid hypertension and diabetes require sustained self-management support and multidisciplinary care. Although generative artificial intelligence may support chronic disease education, quantitative evidence on the clinical quality and actionability of its outputs remains limited. This cross-sectional study compared the quality, readability, and actionability of patient education materials generated by 16 large language models.

Objective:

This study aimed to compare the quality, readability, understandability, and actionability of patient education materials generated by 16 large language models for patients with comorbid hypertension and diabetes.

Methods:

We developed an 80-item guideline-based question bank and submitted each item to 16 large language models, yielding 1,280 texts. Under blinded conditions, the texts were independently evaluated using the Patient Education Materials Assessment Tool for Printable Materials, the Ensuring Quality Information for Patients, 36-item version, the Global Quality Score, and seven readability formulas. Kruskal–Wallis tests, extended Bayesian information criterion graphical least absolute shrinkage and selection operator partial correlation network analysis, and latent profile analysis were performed.

Results:

Median understandability on the Patient Education Materials Assessment Tool for Printable Materials was moderate to high (75.0), whereas actionability remained low (20.0). Network analysis revealed a trade-off between information quality and accessibility: longer texts tended to score higher on the Ensuring Quality Information for Patients, 36-item version but imposed a greater reading burden, and higher Simple Measure of Gobbledygook scores were associated with lower actionability. Latent profile analysis identified five latent profiles; only 19.2% of texts were classified as “high-quality, comprehensive-guidance type.” Profile distribution differed significantly across platforms (p < 0.001), with actionability showing the clearest between-profile separation.

Conclusions:

Mainstream large language models can explain core medical knowledge for this comorbidity, but they remain limited in generating explicit, actionable behavioral guidance. More comprehensive content often comes at the cost of readability. Future digital nursing tools should incorporate multidisciplinary collaboration and behavior-change theory to move generative artificial intelligence from information delivery toward clinically actionable intervention.