JMIR Publications

ABSTRACT

Background:

Natural language processing (NLP) can extract meaningful information from clinical notes. However, human annotation is time-consuming, costly, and scarce data poses a challenge.

Objective:

To explore a semantic similarity search approach to extract exemplars of stigmatizing and positive language in obstetric clinical notes.

Methods:

We used electronic health records data from labor and birth admissions at two hospitals in the United States in 2017. We employed a semantic similarity search approach, which used human-annotated exemplars as queries to search for similar exemplars. We extracted the top five candidates with the highest cosine similarities for 200 stratified and randomly selected true exemplars, which were assessed for accuracy.

Results:

We retrieved 1,000 candidates. Average precision was acceptable (≥ 0.7) when candidates with cosine similarity thresholds of 0.75 or higher were included, at which point 69% of candidates accurately represented true cases. Precision was highest (100%) in the following categories: autonomy for birth, power/privilege, questioning patient credibility, and disapproval. Precision was lowest for difficult patient (29.57%) and marginalized identities (33.33%) categories.

Conclusions:

The semantic similarity search approach shows promise in efficiently extracting exemplars while reducing the annotation burden, laying the groundwork for future applications in other domains. Clinical Trial: This is an observational study and does not require trial registration.