JMIR Publications

ABSTRACT

Background:

The utility of Clinical decision support systems (CDSS) in the clinic remains controversial. In this study, we assessed the effects of CDSS integrated with BMJ best practice for real-world diagnosis.

Objective:

The initial goal was to assess the levels of agreement regarding patients’ diagnosis between CDSS integrated with BMJ best practice and resident doctors. The second goal was to understand whether CDSS integrated with BMJ best practice improves the accuracy of admission diagnosis for inpatients and to explore benefits of CDSS integrated with BMJ best practice on the length of patients’ hospital stay.

Methods:

This was a retrospective longitudinal observational study using routinely collected clinical diagnosis data from Electronic Medical Records (EMR). A total of 34, 113 hospitalized patient records were successively selected from December 2016 to February 2019 in 6 clinical departments. The diagnostic accuracy of CDSS was verified prior to its implementation. A self-controlled comparison was then applied to detect the effects of CDSS implementation. Multivariable logistic regression and single-group interrupted time series (ITS) analysis were used to explore the effects of CDSS. The sensitivity analysis was conducted using the subgroup data from January 2018 to February 2019.

Results:

The total accuracy rates of the recommended diagnosis from CDSS were 75.46% in the 1st rank diagnosis, 83.94% in the top 2 diagnosis and 87.53% in the top 3 diagnosis in data before CDSS implementation. Higher consistency was observed between admission diagnosis and discharge diagnosis, shorter confirmed diagnosis times, and shorter hospitalization days after the CDSS implementation (all p<0.001). Multivariable logistic regression analysis showed that the consistency rates after CDSS implementation (OR=1.078, 95% CI: 1.015-1.144) and the proportion of hospitalization time ≤ 7 days (OR=1.688, 95% CI: 1.592-1.789) both increased. The ITS analysis showed that the consistency rates significantly increased by 6.722% (95% CI=2.433-11.012%, p=0.002) after CDSS implementation. The proportion of hospitalization time ≤7 days significantly increased by 7.837% (95% CI=1.798-13.876%, p=0.011). The similar results were obtained in the subgroup analysis.

Conclusions:

The CDSS integrated with BMJ best practice improved the accuracy of clinicians’ diagnosis. And shorter confirmed diagnosis times and hospitalization days were found to be associated with CDSS implementation in retrospective real-world studies. These findings highlight the utility of AI-based CDSS to improve diagnosis efficiency but require confirmation in future RCT studies.