JMIR Publications

ABSTRACT

Background:

Pulmonary embolism (PE) is a critical condition requiring rapid diagnosis to reduce mortality. Extracting PE diagnoses from radiology reports manually is time-consuming, highlighting the need for automated solutions. Advances in natural language processing (NLP), especially transformer models like GPT-4o, offer promising tools to improve diagnostic accuracy and workflow efficiency in clinical settings.

Objective:

To develop an advanced NLP system using GPT-4o for the automatic extraction of PE diagnoses from radiology report impressions, enhancing clinical decision-making and workflow efficiency.

Methods:

Two approaches were developed and evaluated: a fine-tuned Clinical Longformer as a baseline model, and a GPT-4o-based extractor. The Clinical Longformer was trained on a dataset of 1,000 radiology report impressions and validated on a separate set of 200 samples, while the GPT-4o extractor was validated using the same 200-sample set. Post-deployment performance was further assessed on an additional 500 operational records to evaluate model efficacy in a real-world setting.

Results:

GPT-4o outperformed the Clinical Longformer, achieving 100% sensitivity, specificity, and F1 score across both training and post-deployment evaluations. This high level of accuracy supports a reduction in manual review, streamlining clinical workflows and improving diagnostic precision.

Conclusions:

The GPT-4o model provides an effective solution for the automatic extraction of PE diagnoses from radiology reports, offering a reliable tool that aids timely and accurate clinical decision-making. This approach has the potential to significantly improve patient outcomes by expediting diagnosis and treatment pathways for critical conditions like PE. Clinical Trial: NA