JMIR Publications

ABSTRACT

Background:

Potentially inappropriate prescribing in outpatient care contributes to adverse outcomes and healthcare inefficiencies. Clinical decision support systems offer promising solutions, but their effectiveness is often constrained by incomplete medical records.

Objective:

This study aims to evaluate the effectiveness of a machine learning-based CDSS for enhancing diagnostic recommendations and medication appropriateness.

Methods:

This quasi-experimental study was conducted over one year in the outpatient departments of a hospital. The system provided diagnostic recommendations based on machine learning algorithms trained on National Health Insurance data. Outcome measures included alert rates, acceptance rates of diagnostic recommendations, and variability in system performance across specialties. Descriptive and trend analyses were employed to evaluate the system’s effectiveness.

Results:

This study included 438,558 prescriptions from 44 physicians across 23 specialties, involving 125,000 unique patients in the outpatient departments of a regional teaching hospital. MedGuard, embedded with diagnostic recommendations, achieved an overall alert rate of 2.28% and a diagnostic recommendation acceptance rate of 56.55%. All accepted recommendations resulted in actionable changes, including prescription .adjustments or the addition of missing diagnoses. Ophthalmology achieved the highest acceptance rate at 96.59%, while rheumatology, surgery, psychiatry, and infectious diseases, recorded acceptance rates of 0%, 0%, 24.74%, and 35.00%, respectively. Over the year, acceptance rates for potentially inappropriate prescriptions stabilized at 51%, despite increasing prescription volumes.

Conclusions:

This study demonstrates the potential of embedding diagnostic recommendations into alerts within a machine learning-based clinical decision support system to improve diagnostic completeness and support safer outpatient care. Future efforts should refine alerts to align with specialty-specific workflows and validate their effectiveness in diverse clinical settings.