JMIR Publications

ABSTRACT

Background:

Emergency department (ED) triage system to classify and prioritize patients at high risk from less urgent continues to be a challenge.

Objective:

This study, comprising 80,433 patients, aims to develop a machine learning algorithm prediction model of critical care outcome for adult patients using information collected during ED triage and compare the performance with that of the baseline model using Korean Triage and Acuity Scale (KTAS).

Methods:

To predict the need of critical care, we used 13 predictors from triage information: age, gender, mode of ED arrival, time interval between onset and ED arrival, reason of ED visit, chief complaints, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, oxygen saturation and level of consciousness. The baseline model with KTAS was developed using logistic regression and the machine learning model with 13 variables was generated using extreme gradient boosting (XGB) and deep neural network (DNN) algorithms. The discrimination was measured by area under the receiver operating characteristic curve(AUC). The ability of calibration with Hosmer–Lemeshow test and reclassification with net reclassification index (NRI) were evaluated. The calibration plot and partial dependence plot were used in analysis.

Results:

The AUC of the model with the full set of variables (0.833–0.861) was better than that of the baseline model (0.796). The XGB model of AUC 0.861 (0.848, 0.874, 95% CI) showed a higher discriminative performance than DNN model of 0.833(0.819, 0.848). The XGB and DNN models proved better reclassification than the baseline model with positive net reclassification index. The XGB models was well calibrated (Hosmer-Lemeshow test p>0.05); however, the DNN showed poor calibration power (H-L test p<0.001). We further interpreted non-linear association between variables and critical care prediction.

Conclusions:

Our study demonstrated that the performance of the XGB model using initial information at ED triage for predicting patients in need of critical care outperformed the conventional model with KTAS.