JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI)-based clinical decision support systems (CDSSs) may help identify postoperative patients at risk of delayed discharge, but their clinical value depends on both predictive performance and their impact on surgeons' decision-making and efficiency. Evidence on how AI support influences surgeons' assessment of readiness for discharge after gastrointestinal surgery remains limited.

Objective:

To evaluate whether an AI-based CDSS improves surgeons' accuracy and reduces the time required to predict on-schedule discharge after gastrointestinal surgery.

Methods:

We conducted a crossover study between December 2025 and January 2026 using anonymized retrospective clinical records. Surgeons responsible for discharge decisions were recruited. Participants assessed whether each patient could be discharged as scheduled according to the clinical pathway, based on records available through postoperative day three presented in an emulated electronic medical record (EMR) and CDSS. In the control condition, surgeons reviewed the EMR without AI. In the intervention condition, they first reviewed an AI-generated summary and made an initial prediction; they were then shown the AI-predicted probability of on-schedule discharge and allowed to revise their prediction. The order of conditions was randomized. Each surgeon evaluated the same patient cases in both rounds, with an approximately one-month washout period between rounds. The primary outcomes were prediction accuracy and decision time. Secondary exploratory outcomes included surgeon–AI agreement, changes in prediction after viewing the AI output, and confidence. Analyses were performed overall and at the surgeon and patient levels.

Results:

Ten surgeons completed both rounds and were included in the analysis. Fourteen historical patient cases were used. Of the 14 cases, five involved patients who were discharged on or before the scheduled date. Across 83 paired evaluations, prediction accuracy was 80.7% for AI alone, 71.1% for surgeons without AI, and 69.9% for surgeons with AI; differences were not statistically significant in the overall analysis (Cochran’s Q test, p = .132). Median decision time was shorter with AI than without AI (95 s [1st–3rd quartiles, 69–142] vs 120 s [88–174]; median difference, −20 s, Wilcoxon signed-rank test, p = .001). At the patient level, decision time was also significantly shorter with AI (median difference, −30 s, Wilcoxon signed-rank test, p = .012). Surgeons changed their initial prediction after viewing the AI prediction in two of 83 evaluations; in one case, the prediction changed from correct to incorrect.

Conclusions:

In this crossover study using an emulated EMR, we found no evidence that AI support improved prediction accuracy for on-schedule discharge after gastrointestinal surgery, although it was associated with shorter decision time. AI-generated summaries may improve decision-making efficiency by shortening review time. However, occasional shifts toward incorrect AI predictions suggest the need to address automation bias in CDSS implementation.