JMIR Publications

ABSTRACT

Background:

Clinical notes house rich, yet unstructured patient data, making analysis challenging due to medical jargon, abbreviations, and synonyms causing ambiguity. This complicates real-time extraction for decision support tools.

Objective:

We focus on the data curation, technology, and workflow for NER-based Clinical Decision Support tools that distill key entities from notes to guide patient care.

Methods:

We gathered progress, care, radiology, and pathology notes from 5,000 patients, dividing them into five batches of 1,000 patients each. Metrics like notes/reports per patient, sentence count, token size, runtime, CPU, and memory usage were measured per note type. We also evaluated NER and assertion models against ground truth data.

Results:

Using the SparkNLP clinical NER model on 138,250 clinical notes, we observed excellent NER metrics with a peak precision in "procedures" at 0.989 (95% CI: 0.977-1.000) and an accuracy in the assertion model of 0.889 (95% CI: 0.856-0.922). Our analysis highlighted long-tail distributions in notes per patient, note length, and entity density. Progress care notes had notably more entities per sentence than radiology and pathology notes, showing four-fold and sixteen-fold differences, respectively.

Conclusions:

Further research should explore the analysis of clinical notes beyond the scope of our study, including discharge summaries and psychiatric evaluation notes. Recognizing the unique linguistic characteristics of different note types underscores the importance of developing specialized NER models or NLP pipeline setups tailored to each. By doing so, we can enhance their performance across a more diverse range of clinical scenarios.