JMIR Publications

ABSTRACT

Background:

Resident scheduling is a high‑dimensional optimization problem with implications for workload, fatigue risk, and equity. Real‑world evaluations of AI‑based constraint‑optimization in healthcare are limited.

Objective:

To evaluate an AI‑based constraint‑optimization scheduler with a legacy rule‑based scheduler for pediatric residency night‑calls.

Methods:

Retrospective before-after comparative study across two 8‑month periods at a tertiary pediatric center (legacy software: Jan–Aug 2024; AI‑based software: Jan–Aug 2025). The primary analytic unit was resident‑month. Outcomes included calls and weekend calls per resident‑month, undesired exceedances (> 6 calls, > 2 weekend calls monthly), undesirable sequences, overall and per qualification strata fairness (using mean and root mean square difference from equal share - MAE-ES, RMSE-ES), and schedule publication lead time. Resident surveys were analyzed pre‑ and post‑implementation. To account for pre‑implementation trends, an interrupted time series (ITS) segmented regression modeled level and slope changes at implementation; the primary estimand was the immediate level change and a secondary estimand was the average post‑period effect.

Results:

Across 409 legacy and 484 AI resident‑months (2,144 vs 2,185 shifts), mean calls per resident‑month declined from 5.24 to 4.51 (95% CI −0.93 to −0.56; p < 0.001) and weekend calls from 1.54 to 1.30 (95% CI −0.35 to −0.14; p < 0.001). Frequency of exceedances decreased (>6 calls: 19.6% to 4.8%; RR 0.27, 95% CI 0.16–0.44; p < 0.001; >2 weekend calls: 7.8% to 2.9%; RR 0.37, 95% CI 0.18–0.73; p = 0.004). Undesirable sequences were reduced for consecutive weekends (34.5 to 18.8), call–rest–call (73.3 to 26.9), and call–rest–call–rest–call (8.80 to 1.03) events per 100 resident‑months (all p < 0.001). Fairness improved (MAE‑ES 1.20 to 1.01; RMSE‑ES 1.51 to 1.24), particularly among Advanced and Novice strata. Publication lead time increased from 10.6 ± 2.3 to 21.8 ± 1.7 days (U = 0; p = 8.83×10⁻⁴). Pre- and post- implementation questionnaires showed increased satisfaction (6.77 to 8.67, p<0.001), improved perceived publication timeliness (3.28 vs. 4.58, p<0.001), improvement to consecutive night calls (3.15 vs. 4.25 p<0.001), perceived fairness (2.98 vs. 3.56, p=0.012) and assignment clarity (4.38 vs. 4.78 p=0.048). In interrupted time‑series models, mean calls showed an immediate level drop at implementation (−0.71; p < 0.001) and consecutive‑weekend burden improved both immediately (−7.65; p = 0.0068) and on average post‑implementation (−10.18; p = 0.013), with marked gains in schedule lead time at go‑live (+8.96 days; p < 0.001) and sustained thereafter (+7.27; p < 0.001).

Conclusions:

An AI‑based constraint‑optimization scheduler was associated with reduced workload, fewer undesirable sequences, improved fairness, and earlier publication versus a legacy scheduler. New AI-based solutions hold promise for improving healthcare operations and optimize human resource management.