JMIR Publications

ABSTRACT

Background:

Clinician trust in machine-learning-based clinical decision support systems (CDSS) for predicting in-hospital deterioration (a type of predictive CDSS) is essential for adoption. Evidence shows that clinician trust in predictive CDSSs is influenced by perceived understandability and perceived accuracy.

Objective:

To explore the phenomenon of clinician trust in predictive CDSSs for in-hospital deterioration by confirming and characterizing factors known to influence trust (understandability and accuracy), uncovering and describing other influencing factors, and comparing nurses’ and prescribing providers’ trust in predictive CDSSs.

Methods:

We followed qualitative descriptive methodology, conducting directed deductive and inductive content analysis of interview data. Directed deductive analyses were guided by the Human-Computer Trust conceptual framework. Semi-structured interviews were conducted with nurses and prescribing providers (physicians, physician assistants, or nurse practitioners) working with a predictive CDSS at Brigham and Women’s Hospital and Newton Wellesley Hospital.

Results:

Seventeen clinicians were interviewed. Concepts from the Human-Computer Trust conceptual framework, perceived understandability and perceived technical competence (i.e., perceived accuracy), were found to influence clinician trust in predictive CDSSs for in-hospital deterioration. Concordance between clinicians’ impressions of patients’ clinical status and system predictions influenced clinicians’ perceptions of system accuracy. Understandability was influenced by system explanations, both global and local, as well as training. Three additional themes emerged from the inductive analysis. One, perceived actionability, captured the variation in clinicians’ desires for predictive CDSSs to recommend a discrete action. The second, evidence, describe the importance of both macro (scientific) and micro (anecdotal) evidence for fostering trust. The final theme, equitability, described fairness in system predictions. Findings were largely similar between nurses and prescribing providers.

Conclusions:

While there is a perceived tradeoff between machine-learning-based CDSS accuracy and understandability, our findings confirm that both are important for fostering clinician trust in predictive CDSSs for in-hospital deterioration. We found that reliance on the predictive CDSS in the clinical workflow may influence clinicians’ requirements for trust. Future research should explore the impact of reliance, optimal explanation design for enhancing understandability, and the role of perceived actionability in driving trust.