Currently submitted to: JMIR AI
Date Submitted: May 12, 2026
Open Peer Review Period: May 14, 2026 - Jul 9, 2026
(closed for review but you can still tweet)
NOTE: This is an unreviewed Preprint
Warning: This is a unreviewed preprint (What is a preprint?). Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn (a note "no longer under consideration" will appear above).
Peer review me: Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period (in this case, a "Peer Review Me" button to sign up as reviewer is displayed above). All preprints currently open for review are listed here. Outside of the formal open peer-review period we encourage you to tweet about the preprint.
Citation: Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author).
Final version: If our system detects a final peer-reviewed "version of record" (VoR) published in any journal, a link to that VoR will appear below. Readers are then encourage to cite the VoR instead of this preprint.
Settings: If you are the author, you can login and change the preprint display settings, but the preprint URL/DOI is supposed to be stable and citable, so it should not be removed once posted.
Submit: To post your own preprint, simply submit to any JMIR journal, and choose the appropriate settings to expose your submitted version as preprint.
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Development and Preliminary Validation of CLEAR: A Framework for Evaluating Patient-Friendly AI-Generated Clinical Documentation
ABSTRACT
Background:
Generative artificial intelligence (GenAI) is increasingly used to produce patient-friendly clinical documentation, yet evaluation of these outputs remains inconsistent and difficult to scale. Patient-friendliness is commonly reduced to narrow readability metrics, such as Flesch-Kincaid grade level, without accounting for clinical accuracy, completeness, or the patient perspective. No standardized framework exists to evaluate the quality and safety of AI-generated patient-friendly documentation across document types or the full documentation lifecycle.
Objective:
To develop and preliminarily validate CLEAR (Clinical Language Evaluation and AI Documentation Review), a theoretically grounded evaluation framework for AI-generated patient-friendly clinical documentation across the generation, review, and monitoring stages of the AI documentation lifecycle.
Methods:
CLEAR was developed using Messick's validity framework across four stages: content validation, response process, internal structure, and consequences. Domains were identified through a targeted literature review and reviewed by a panel of six clinical and operational experts. An iterative, consensus-based process involving four board-certified internists across 10 rounds refined domain definitions and scoring instructions. Inter-rater reliability was assessed on 50 AI-generated patient-friendly discharge summaries using Cohen's kappa and Gwet's AC1 for binary domains and intraclass correlation coefficients (ICC) and Gwet's AC2 for continuous domains. Additionally, 19 semi-structured stakeholder interviews with clinicians, informaticists, institutional leaders, and patient education experts explored operational needs and implementation contexts.
Results:
CLEAR comprises five domains for evaluating patient-friendly AI documentation: readability, understandability, patient-centeredness, accuracy, and completeness. Inter-rater reliability was good to almost perfect across all subjectively scored domains per Gwet's agreement coefficients. Stakeholder interviews independently identified three operational gaps aligned with the CLEAR lifecycle: lack of structured guidance for prompt engineering, subjectivity in human review, and absence of scalable monitoring infrastructure, directly validating the framework's real-world relevance. CLEAR was applied across three illustrative implementation contexts: prompt engineering for patient-friendly echocardiogram reports, structured human review of discharge summaries, and development of LLM-as-judge automated monitoring tools.
Conclusions:
CLEAR provides a preliminarily validated evaluation framework designed to span the full AI documentation lifecycle, from prompt engineering through human review to automated monitoring. By conceptualizing patient-friendliness as a multidimensional construct that integrates communication quality with patient safety, CLEAR offers practical infrastructure for consistent and scalable governance of patient-facing AI documentation in healthcare systems.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.