JMIR Publications

ABSTRACT

Background:

Management of abdominal aortic aneurysm (AAA) requires imaging surveillance to evaluate aneurysm size serially. Natural language processing (NLP) has been previously developed to identify patients with AAA retrospectively. However, there are no prior reported prospective studies using NLP to identify AAA patients in real-time from radiology reports

Objective:

To develop and validate a rule-based natural language processing (NLP) algorithm for near real-time automatic extraction of AAA diagnosis from radiology reports for prospective case identification.

Methods:

The AAA-NLP algorithm was developed and deployed on the electronic health record big data infrastructure for near real-time processing of radiology reports from 5/1/2019 to 9/30/2020. NLP extracted named entities for AAA case identification and classified subjects as cases and controls. The reference standard to assess algorithm performance was manual review of processed radiology reports by a trained physician and certified cardiologist following standardized criteria. Reviewers were blinded to diagnosis of each subject. The AAA-NLP algorithm was refined in three successive iterations. For each iteration the AAA-NLP algorithm was modified based on performance compared with the reference standard.

Results:

120 radiology reports were randomly selected for each iteration; a total of 360 reports were reviewed. At each iteration, the AAA-NLP algorithm performance improved. The algorithm identified AAA cases in near real-time with high positive predictive value (98 %), sensitivity (95 %), specificity (98 %), F1 score (97 %), and accuracy (97 %).

Conclusions:

Implementation of NLP for identification of AAA cases from radiology reports in near real-time with high performance is feasible. This prospective real-time NLP technique will generate automated input for patient care, quality projects, and clinical decision support tools for the management of AAA patients.