JMIR Publications

ABSTRACT

Background:

Early warning score (EWS) systems are widely used for identifying patients who are at the highest risk of deterioration to assist clinical decision-making. This could facilitate early intervention, and consequently improve patient outcomes. For example, the National Early Warning Score (NEWS), which is recommended by the Royal College of Physicians in the United Kingdom, assigns scores to routine vital signs based on pre-defined alerting thresholds. However, there is limited evidence on the reliability of such scores across patient cohorts in the United Arab Emirates (UAE).

Objective:

Our aim in this study is to propose an interpretable data-driven model for improved deterioration prediction in an in-patient cohort in the United Arab Emirates (UAE).

Methods:

We conduct a retrospective cohort study using a real-world dataset consisting of 25,853 in-patient encounters from 16,729 unique patients, collected between April 2015 and August 2021 at a large multi-specialty hospital in Abu Dhabi, UAE. We define an adverse event as the composite outcome of admission to the intensive care unit or mortality. Based on seven routine vital-sign measurements, we assess the performance of the NEWS in detecting deterioration within 24 hours using the Area Under the Receiver Operating Characteristic curve (AUROC). We also develop and evaluate several machine learning models, including logistic regression, gradient boosting model, and a feed-forward neural network.

Results:

In a held-out test set of 2,534 encounters with 74,630 observation sets, NEWS achieves an overall AUROC of 0.682 (95% CI 0.670-0.695). In comparison, the best-performing machine learning models, which are the neural network and the gradient boosting model, achieve an AUROC of 0.735 (95% CI 0.723-0.746) and 0.736 (95% CI 0.724-0.748), respectively. Our interpretability results highlight the importance of respiratory rate and level of consciousness in predicting patient deterioration.

Conclusions:

While traditional EWS systems are the dominant form of deterioration prediction models in clinical practice today, we strongly recommend the development and use of cohort-specific machine learning models as an alternative. This is especially important in external patient cohorts that were unseen during model development.