JMIR Publications

ABSTRACT

Background:

Computerized decision support systems (CDSS) are increasingly adopted in healthcare to optimize resources and streamline patient flow. However, they often lack scientific validation against standard medical care.

Objective:

The purpose of the study was to assess the performance, safety and usability of a CDSS in a university hospital Emergency Department (ED) setting in Kuopio, Finland.

Methods:

Patients entering the ED were voluntarily asked to participate in the study. Patients aged 17 years or less, patients with cognitive impairments, and patients entering the unit in an ambulance or with the need for immediate care were excluded. Patients completed the CDSS online form and usability questionnaire when waiting for the triage nurse’s evaluation. The CDSS data were anonymized and did not affect the patients' usual evaluation or treatment. Retrospectively, two medical doctors (MDs) evaluated the urgency of each patient’s condition using the triage nurse’s information, and urgent and non-urgent groups were created. The ICD-10 diagnoses were collected from the electronic health records. Usability was assessed using a positive SUS questionnaire.

Results:

In total, our analyses included 248 patients. Regarding urgency, mean sensitivities were 85% and 19%, respectively, for urgent and non-urgent cases when assessing the performance of CDSS evaluations compared to MDs. The sensitivities were 85% and 35%, respectively, when comparing the evaluations between MDs. Our CDSS did not miss any cases that were evaluated to be emergencies by MDs, thus all emergency cases evaluated by MDs were evaluated as either urgent or emergency by the CDSS. In differential diagnosis, the CDSS had an exact match accuracy of 46%. The usability was good with a mean (SD) SUS score of 78.2 (16.8).

Conclusions:

In a university hospital emergency room setting with a large real-world population, our CDSS was found to be equally as sensitive in urgent patient cases as physicians, and to have an acceptable differential diagnosis accuracy, with good usability. These results suggest that this CDSS can be safely assessed further in a real-world setting. A CDSS could accelerate triage by providing the patient-provided data in advance of their initial consultation, and suggest distributing patients into urgent and non-urgent cases upon emergency room arrival. Clinical Trial: NCT04577079