JMIR Publications

ABSTRACT

Background:

Surgical consent forms must convey critical information, yet their complex language can limit patient comprehension. Large language models (LLMs) may improve readability, but evidence of their impact on content preservation is lacking in non-English contexts.

Objective:

This study evaluates the impact of LLM-assisted editing on the readability and content quality of surgical consent forms in Korean, focusing on standardized liver resection consent documents across multiple institutions.

Methods:

Standardized liver resection consent forms were collected from seven South Korean medical institutions and simplified using ChatGPT-4o. Readability was assessed using KReaD and Natmal indices, while text structure was evaluated based on character count, word count, sentence count, words per sentence, and difficult word ratio. Content quality was analyzed across four domains—Risk, Benefit, Alternative, and Overall Impression—using evaluations from seven liver resection specialists. Statistical comparisons were conducted using paired t-tests, and a linear mixed-effects model (LME) was applied to account for institutional and evaluator variability.

Results:

AI-assisted editing significantly improved readability, reducing the KReaD score from 1777 ± 28.47 to 1335.6 ± 59.95 (p<0.001) and the Natmal score from 1452.3 ± 88.67 to 1245.3 ± 96.96 (p=0.007). Sentence length and difficult word ratio decreased significantly, contributing to increased accessibility. However, content quality analysis showed a decline in risk description scores (2.29 ± 0.47 before vs. 1.92 ± 0.32 after, p=0.0549) and overall impression scores (2.21 ± 0.49 before vs. 1.71 ± 0.64 after, p=0.134). The LME confirmed significant reductions in risk descriptions (β₁ = -0.371, p=0.012) and overall impression (β₁ = -0.500, p=0.025), suggesting potential omissions in critical safety information. Despite this, qualitative analysis indicated that evaluators did not find explicit omissions but perceived the text as overly simplified and less professional.

Conclusions:

While LLM-assisted surgical consent forms significantly enhance readability, they may compromise certain aspects of content completeness, particularly in risk disclosure. These findings highlight the need for a balanced approach that maintains accessibility while ensuring medical and legal accuracy. Future research should include patient-centered evaluations to assess comprehension and informed decision-making, as well as broader multilingual validation to determine LLM applicability across diverse healthcare settings. Clinical Trial: N/A