JMIR Publications

ABSTRACT

Background:

Explainable user interfaces (XUIs) for AI-based clinical decision support systems aim to make model predictions interpretable for clinicians. Two paradigms have emerged: Explanatory XUIs deliver static feature-importance summaries with clinical interpretation text, whereas Exploratory XUIs provide interactive, multi-modal model inquiries. Comparisons of both paradigms that include deliberately incorrect AI predictions remain limited.

Objective:

To compare an Explanatory and an Exploratory XUI for AI-supported sepsis risk assessment on user experience, decision behavior, and reliance on incorrect predictions.

Methods:

In a counterbalanced within-subjects study conducted between April and June 2025, N=17 clinicians (6 physicians, 7 nurses/paramedics, 4 other healthcare professionals) with varying intensive care unit (ICU) experience evaluated three patient cases per XUI condition using SepsisVision, a sepsis risk prediction system, each containing two correct (TP, TN) and one incorrect (FP or FN) prediction. Outcomes covered user experience (SUS usability, NASA-TLX workload, task time, trust, explanation satisfaction), decision behavior (accuracy, confidence, automation bias, appropriate skepticism, appropriate reliance), and moderation by clinical expertise. Two prespecified test families, five user-experience and six decision-level comparisons, were Holm-Bonferroni corrected. Baseline, expertise and correlational analyses were not tested and are reported as effect sizes with 95% CIs. Think-aloud protocols were analyzed thematically.

Results:

The Explanatory XUI achieved higher usability (SUS M=85.9 vs. M=71.3, adj. P=.001, r=.898), lower cognitive workload (adj. P=.025, r=.659), and shorter task time (adj. P=.019, r=.687). Post-AI decision accuracy did not differ between conditions (Explanatory M=72.5% vs. Exploratory M=74.5%; paired difference +2.0 percentage points, 95% CI −15.7 to +17.6). Participants moved toward the AI regardless of correctness, so accuracy rose on AI-correct cases but fell on AI-incorrect cases, with the largest drop of 40 percentage points on the Exploratory XUI (FP). Automation bias occurred in both conditions (Explanatory 4/17, 24%; Exploratory 5/17, 29%), and appropriate skepticism was higher with the Explanatory XUI overall (65% vs. 47%). Both were moderated by experience. ICU-experienced participants resisted the incorrect recommendation under both interfaces (86%), whereas less-experienced participants fell from 50% to 20% under the Exploratory XUI. Sepsis experience correlated with appropriate skepticism in that condition (r=+.82, 95% CI .66–.91). Mean confidence rose after AI exposure under the Exploratory XUI (3.29 to 3.67), without a matching accuracy gain. Participants independently proposed a hybrid design.

Conclusions:

No difference in average performance was detected between XUI types, but the two interfaces diverged by clinical expertise. The Exploratory XUI was associated with reduced appropriate skepticism among less-experienced clinicians and with higher post-AI confidence unmatched by higher accuracy, consistent with miscalibrated reliance, whereas the skepticism of experienced clinicians was unchanged. Because most predictions were correct, this risk was invisible in aggregate accuracy. For mixed-expertise settings, a structured low-burden default with progressive access to interactive features is recommended.