JMIR Publications

ABSTRACT

Background:

While clinical decision support alerts are effective reminders of best practices, their effectiveness is blunted by clinicians who fail to respond to an overabundance of inappropriate alerts. An EHR- integrated machine learning (ML) algorithm is a potentially powerful tool to increase the signal-to-noise ratio of CDS alerts and positively impact clinician interaction with these alerts in general.

Objective:

This study describes the development and implementation of a ML based signal-to-noise optimization system (SmartCDS) to increase the “signal” of alerts by decreasing the volume of low value herpes zoster (shingles) vaccination alerts.

Methods:

We built and deployed SmartCDS that builds personalized user activity profiles to suppress shingles vaccination alerts unlikely to yield clinician interaction. We extracted all shingles alert records from January 2017 to March 2019 from our EHR system, including 327,737 encounters, 780 providers, and 144,438 patients.

Results:

During six weeks of pilot deployment, the SmartCDS system suppressed an average of 43.7% of 35,315 potential shingles alerts (appointments) while maintaining stable counts of weekly shingles orders (326.3 model on vs. 331.3 model off, P=.38) and weekly user-alert interactions (1118.3 model on vs. 1166.3 model off, P=.20).

Conclusions:

We demonstrated that an automated, ML-based method and data architecture to suppress alerts is feasible without detriment to overall order rates. This work is the first alert suppression machine learning based model deployed in practice and serves as foundational work in encounter-level customization of alert display to maximize effectiveness.