Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Oct 30, 2019
Date Accepted: Feb 21, 2020
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
The development, implementation, and evaluation of a personalized machine learning algorithm for clinical decision support: A case study with shingles vaccination.
ABSTRACT
Background:
While clinical decision support alerts are effective reminders of best practices, their effectiveness is blunted by clinicians who fail to respond to an overabundance of inappropriate alerts. An EHR- integrated machine learning (ML) algorithm is a potentially powerful tool to increase the signal-to-noise ratio of CDS alerts and positively impact clinician interaction with these alerts in general.
Objective:
This study describes the development and implementation of a machine learning (ML) based signal-to-noise optimization system to increase the “signal” of alerts by decreasing the volume of low value herpes zoster (shingles) vaccination alerts.
Methods:
During six weeks of pilot deployment, the signal-to-noise optimization system suppressed an average of 45% of daily shingles alerts while maintaining stable counts of daily shingles orders (46.6 model on vs. 47.3 model off) and user-alert interactions (159.7 model on vs. 166.0 model off).
Results:
During six weeks of pilot deployment, the signal-to-noise optimization system suppressed an average of 45% of daily shingles alerts while maintaining stable counts of daily shingles orders (46.6 model on vs. 47.3 model off) and user-alert interactions (159.7 model on vs. 166.0 model off).
Conclusions:
We demonstrated that an automated, ML-based method and data architecture to suppress alerts is feasible without detriment to overall order rates. This work is the first alert suppression machine learning based model deployed in practice and serves as foundational work in encounter-level customization of alert display to maximize effectiveness.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.