JMIR Publications

ABSTRACT

Background:

Studies have shown that patients have difficulty understanding medical jargon in electronic health record (EHR) notes. In creating the NoteAid dictionary of medical jargon for patients, a panel of medical experts selected terms they perceived as needing definitions for patients.

Objective:

To determine whether experts and laypeople agree on what constitutes jargon.

Methods:

Using an observational study design, we evaluated the agreement between 6 medical experts and 270 untrained laypeople (crowdsource workers) in jargon term identification in 20 sentences from EHR notes. They contained 325 potential jargon terms.

Results:

There was good agreement among medical experts (Fleiss’ kappa = 0.781, 95% CI: 0.753–0.809) and fair agreement among laypeople (Fleiss’ kappa = 0.590, 95% CI: 0.589–0.591). The medical experts had high sensitivity (91.7%, 95% CI: 90.1–93.3%) and specificity (88.2%, 95% CI: 86.0–90.5%) in identifying jargon terms as determined by the laypeople. The proportion of terms marked as jargon by different demographic groups among the laypeople ranged from 17.7% (95% CI: 10.7–24.7%) to 30.9% (95% CI: 28.1–33.8%).

Conclusions:

We showed that medical experts could accurately identify jargon terms for annotation that would be useful for laypeople.