Currently submitted to: JMIR Diabetes
Date Submitted: Feb 19, 2026
Open Peer Review Period: Feb 27, 2026 - Apr 24, 2026
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Assessing Quality, Readability, and Transparency of Online and Artificial Intelligence-Generated Information on Type 2 Diabetes Mellitus: A Cross-Sectional Study
ABSTRACT
Background:
Type 2 Diabetes Mellitus (T2DM) affects approximately 590 million people worldwide, and its management relies heavily on patient education. With the emergence of online health information and Artificial Intelligence Large Language Models (AI LLMs), patients are increasingly sourcing medical information independently.
Objective:
This study compares the quality, readability and reliability between traditional online resources and AI-generated information related to T2DM.
Methods:
Four predefined search terms were entered into three major search engines (Google, Yahoo, and Bing), and the top 20 search results were retrieved. Patient information AI-generated leaflets (AIGLs) were produced using a standardised prompt across four AI LLMs (ChatGPT, Gemini, DeepSeek, and Grok). Information quality was assessed using the DISCERN score and was calculated by the author and ChatGPT. The JAMA benchmark was used to measure reliability and transparency. The Flesh Reading Ease Score (FRES) and Flesh-Kincaid Grade Level (FKGL) were used to determine the readability and comprehension.
Results:
Eighty websites and four AIGLs were analysed, with author-rated mean DISCERN scores of 43.6 (±10.9) and 43.8 (±2.986), mean JAMA Benchmark scores of 2.74 (±0.965) and 0, mean FRES of 50.6 (±14.4) and 48.9 (±9.16), and mean FKGL scores of 8.66 (±2.23) and 8.3 (±1.92), respectively. The ChatGPT-rated mean DISCERN scores for websites and AIGLs were 58.5 (±11.5) and 61.0 (±2.94), respectively.
Conclusions:
Given the high prevalence of T2DM, both traditional online and AI-generated T2DM resources demonstrate suboptimal quality, accessibility, and transparency. Increasing patient reliance on digital health information calls for improved readability standards and stronger safeguards for AI-generated content. The landscape of medical consultations is evolving, with patients increasingly presenting with preconceived notions based on online health information; hence, healthcare professionals should adapt to this shift.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.