JMIR Publications

ABSTRACT

Background:

The rapid integration of Artificial Intelligence (AI) in healthcare, particularly for diabetes risk prediction, holds significant promise for improving patient outcomes. However, the "black-box" nature of deep learning models remains a primary barrier to clinical adoption. While the field of Explainable AI (XAI) has introduced numerous techniques to enhance transparency, clinical adoption continues to be stifled by a fundamental reliance on clinician trust. Despite an increase in research focusing on technical interpretability, there remains a critical measurement gap: existing literature frequently fails to evaluate or quantify how specific XAI techniques such as SHAP (Shapley Additive Explanations) and Counterfactual Analysis actually influence the multi-dimensional constructs of trust held by medical professionals. Without mapping technical explanations to validated human-centric trust models, the clinical utility of XAI remains theoretical rather than evidentiary.

Objective:

The primary objective of this study is to evaluate the impact of different XAI modalities on clinician trust within the context of diabetes prediction. Specifically, this research seeks to operationalize and map SHAP (feature attribution) and Counterfactual (actionable "what-if" scenarios) explanations to the Asan et al (2020) Trust Framework. The study aims to determine how these specific explainability techniques influence the clinicians trust. Ultimately, this work aims to provide a structured approach for measuring the effectiveness of XAI in fostering the professional trust required for real-world clinical deployment.

Methods:

The study was conducted in four distinct phases. First involved a foundational exploration of trust measurement by integrating psychological, behavioral, and cognitive aspects of human-AI interaction. Next, these general trust constructs were aligned with specific clinical trust measurement requirements to filter out essential trust measurement metrics. Phase 3 involved the development of a structured clinician trust measurement questionnaire, specifically tailored to evaluate the SHAP and Counterfactual explanations generated by decision-support model of diabetes prediction model of our previous work titled "Enhancing Clinical Trust in Diabetes Prediction: A Multi-Directional Counterfactual and SHAP-based Decision Support Model". Phase 4 consisted of primary data collection from practicing clinicians using a hybrid approach of Google Forms and hard-copy questionnaires. Participants evaluated the model’s explanations against real patient profiles collected from Hiwot Fana Comprehensive Specialized University Hospital. Finally, the collected data were analyzed to quantify the relationship between XAI modalities and the filtered clinical trust measurements.

Results:

Clinicians reached a positive consensus on core diagnostic pillars, specifically where SHAP-driven feature importance and Counterfactual Analysis aligned with clinical intuition. This transparency bolstered decision certainty and appropriate reliance. However, a "trust ceiling" persisted regarding outlier resilience, as clinicians remained skeptical of the model’s performance with atypical lab results and data asymmetry. While institutional accreditation was the strongest driver of overarching trust, cloud-connectivity bottlenecks hindered platform suitability. Qualitatively, practitioners advocated for "Communicative AI," favoring structured written summaries over abstract visual plots for faster bedside interpretation.

Conclusions:

Clinician trust in diabetes AI is multi-dimensional and conditional, rather than binary. While explainability (XAI) fosters informed certainty, Institutional Accreditation remains the primary catalyst for professional adoption. The disparity between high logical trust and low outlier resilience suggests that clinicians currently view AI as a "routine collaborator" rather than a surrogate for complex edge-case reasoning. To bridge the remaining trust gaps, future developments must move beyond abstract visualizations toward "Communicative AI" utilizing natural language summaries and edge-computing architectures to ensure the tool is both medically intuitive and infrastructure-resilient in resource-limited settings. Clinical Trial: First