JMIR Publications

ABSTRACT

Background:

Surgical site infections (SSIs) affect 160,000 to 300,000 patients annually, increasing postoperative mortality, causing significant complications, and incurring $3.5 to $10 billion in excess costs each year. Effective SSI surveillance can inform strategies to mitigate these outcomes. Traditional SSI surveillance methods, primarily manual chart reviews, are costly and labor-intensive.

Objective:

This study aimed to evaluate whether an automated SSI surveillance system built using newer natural language processing (NLP) methods and deep learning can outperform previous approaches and whether such an approach can enable more efficient infection surveillance.

Methods:

Our dataset comprised of approximately 30,000 surgical cases from University of Washington Medical Center (UWMC) and Harborview Medical Center (HMC). Data from UWMC were captured for the National Surgical Quality Improvement Program (NSQIP) and data from HMC were captured for National Healthcare Safety Network (NHSN). Electronic health record (EHR) data for each surgical case included structured data, pertaining to surgical procedure characteristics, laboratory values, and antibiotic administration, and clinical text notes for a surgical case from 7 days before to 90 days after surgery. Using this data, we used a myriad of machine learning (ML) approaches on the task of SSI prediction. We reported the following performance metrics: F1- score, precision (positive predictive value), recall (sensitivity), AUPRC, and Precision at 0.9 Recall for each machine learning approach.

Results:

Our study demonstrates that incorporating multimodal information from the EHR like contextual information in clinical text and temporal information in laboratory values enhances SSI prediction performance (structured+clinical text F1: 0.68, structured only F1: 0.54; P<0.001, structured+clinical text+temporal F1: 0.70, structured+clinical text F1: 0.68; P<0.001). Leveraging deep learning and large language models (LLMs) also enhances performance (deep learning + LLM F1: 0.70, state-of-the-art (SOTA) rule-based system F1: 0.43; P<0.001). The optimal approach combines generalist foundation models for specialized tasks like text summarization with other deep learning techniques for clinical text and temporal data processing. This system outperforms published baselines, achieving a precision of 0.38 at 0.9 recall, demonstrating its potential for efficient, data-driven SSI surveillance. Additionally, we examine factors influencing model predictions.

Conclusions:

Automated surveillance approaches, in particular deep learning approaches, in combination with voluminous, multimodal data from the EHR can enable more efficient infection surveillance process. This has the potential to increase the quantity of SSI surveillance data available to guide interventions aimed at reducing SSI rates.