JMIR Publications

ABSTRACT

Background:

Health literacy, the ability to understand and act on health information, is critical for patient outcomes and healthcare system efficiency. While plain language guidelines enhance text-based communication, audio-based health information remains underexplored, despite the growing use of virtual assistants and smart devices in healthcare. Traditional readability formulas, such as Flesch-Kincaid, provide limited insights into the complexity of health-related texts and fail to address challenges specific to audio formats. Factors like syntax and semantic features significantly influence comprehension and retention across modalities.

Objective:

This study investigates features that affect comprehension of medical information delivered via text or audio formats. We also examine existing readability formulas and their correlation with perceived and actual difficulty of health information for both modalities.

Methods:

We developed a parallel corpus of health-related information that differed in delivery format: text or audio. We used text from BMJ Lay Summary (N = 193), WebMD (N = 40), Patient Instruction (N = 40), Simple Wikipedia (N = 243), and BMJ Journal (N = 200). Participants (N = 487) read or listened to a health text and then completed a questionnaire evaluating perceived difficulty of the text measured using a 5-point Lickert scale and actual difficulty measured using multiple-choice and true-false questions (comprehension) as well as free recall of information (retention). Questions were generated by ChatGPT 4.0. Underlying syntactic, semantic, and domain-specific features, as well as common readability formulas, were evaluated for their relation to information difficulty.

Results:

In general, the text versions were perceived as easier than the audio versions. The features of the underlying text were related to both perceived and actual difficulty. Longer texts were perceived to be more difficult in text than audio, while free recall decreased with longer texts in both modalities. Higher content word frequency was associated with lower perceived difficulty in audio and improved free recall results for text. Verb-heavy content was easier to comprehend, especially in audio, while noun- and adjective-heavy content increased difficulty. Finally, readability formulas are found ineffective in assessing information difficulty, except for the perceived difficulty in the text condition.

Conclusions:

Text was more effective for conveying complex health information, but audio can be suitable for easier content. In addition, several textual features affect information comprehension and retention for both modalities. Finally, existing readability formulas did not explain actual difficulty. This study highlighted the importance of tailoring health information delivery to content complexity by using appropriate style and modality.