JMIR Publications

ABSTRACT

Background:

The successful implementation of decision support systems promises to enhance high quality care. However, the successful implementation of a clinical decision support system (CDSS) depends on the users’ acceptance and adoption of the system. A machine learning based CDSS to assist primary care professionals treating urinary tract infections was implemented and usability usefulness was assessed through a questionnaire.

Objective:

The primary goal of this study was to assess the system's usability by examining users’ experiences with the software. The secondary goal of interest is to assess users’ attitudes towards evidence- based practices and innovations in healthcare.

Methods:

In collaboration with NIVEL and LUMC, Pacmed developed the CDSS. The cohort was mostly recruited on caregroup level, and all the practices within a caregroup were obliged to participate. Health insurers funded the research partly. The practices participated in the implementation study for a period of four months. A survey was constructed using the unified theory of technology acceptance model (UTAUT) and sent to 265 general practitioners and assistants shortly after the implementation period. Furthermore, usage data was analysed.

Results:

Thirty out of 34 participating practices that used the software (88%) submitted at least one response with practices submitting on average 2.23 responses (SD = 1.43). The CDSS was used continuously by all practices. 31 practices continued using the tool throughout the pilot period, with a 9% dropout in the first 8 weeks. Seventy percent of respondents trusted the tool’s output, and 73% found it understandable how the algorithm came to predictions. Sixty- seven percent of respondents indicated that the information provided seemed useful in addition to the available guidelines, and 51% agreed that it supported their decision-making process. However, the majority of respondents could not positively estimate whether using the tool improved patient care (44%) or whether the use improved patient outcomes (64%). Forty-eight percent of respondents found the software easy to integrate in their clinical workflow.

Conclusions:

The CDSS was perceived as trustworthy and easy to use. However, users were unable to determine whether the CDSS improved patient outcome. Also, the study could have benefitted from including assistants as well as general practitioners more in the design phase of the software. Assistants play an important role in the treatment of urinary tract infections, and to design the software that fits the current process of care well, seems to have potential of positive effects on the perceived time investment of the use of the tool. Lastly, all the users seem to be very highly motivated to contribute to further research in this field and indicate to be willing to embrace change in the way healthcare is provided. This could, however, also signal selection bias in our study. Clinical Trial: Trial Registration: ClinicalTrials.gov NCT04408976.