JMIR Publications

ABSTRACT

Background:

Many promising Artificial Intelligence (AI) and Computer-Aided Detection and Diagnosis (CAD) systems have been developed in lab settings, but few have been successfully integrated into clinical practice. This is partially due to a lack of user-centered design of AI-CAD systems.

Objective:

To assess the impact of different onboarding tutorials and different levels of AI model explainability on radiologists’ trust in AI and the utilization of AI recommendations in lung nodule assessment on CT scans.

Methods:

Twenty radiologists from 7 Dutch medical centers performed lung nodule assessment on CT scans under different conditions in a simulated use study as part of a 2x2 repeated measures experimental design. Two types of AI onboarding tutorials (reflective vs. informative) and two levels of AI output (black box vs. explainable) were designed. Participating radiologists first received an onboarding tutorial, which was either informative or reflective. Subsequently, each radiologist assessed 7 CT scans, first without AI recommendations. AI recommendations were shown to the radiologist, and they could adjust their initial assessment if they found it appropriate. Half of the participants received the recommendations via black box AI output and the other half received explainable AI output. Mental model and psychological trust were measured before onboarding, after onboarding and after assessing the 7 CT scans. We recorded whether radiologists changed their assessment on found nodules, their malignancy prediction, and follow-up advice for each CT assessment. In addition, we analyzed whether radiologists’ trust in their assessments had changed based on the AI recommendations.

Results:

Both variations of onboarding tutorials resulted in a significantly better understanding of the capabilities and limitations of the AI-CAD system (informative p=0.011 and reflective p=0.007). After using AI-CAD, the psychological trust significantly decreased for the group with explainable AI output (p=0.021). Based on the AI recommendations, radiologists changed the number of reported nodules in 27/140 assessments, the malignancy prediction in 32/140 assessments and the follow-up advice in 12/140 assessments. The changes were mostly an increased number of reported nodules, a higher estimated probability of malignancy, and earlier follow-up. The radiologists’ confidence in their found nodules changed in 82/140 assessments, in their estimated probability of malignancy in 50/140 assessments, and in their follow-up advice in 28/140 assessments. These changes were predominantly increases in confidence. The number of changed assessments and changed radiologists’ confidence did not significantly differ between the groups that received different onboarding tutorials and AI outputs.

Conclusions:

Onboarding tutorials help radiologists in getting a better understanding of the AI-CAD and facilitate formation of a correct mental model of the AI-CAD. If AI explanations do not consistently substantiate the probability of malignancy across patient cases, this can impair radiologists' trust in the AI-CAD system. Radiologists’ confidence in their assessments was improved by using the AI recommendations. Clinical Trial: NA