JMIR Publications

ABSTRACT

Background:

Emergency Departments (EDs) are vital access points for urgent care, yet persistent overcrowding in Canadian healthcare systems compromises service quality and efficiency. Urban tertiary care centres face increasing strain from rising patient volumes and limited inpatient bed availability. Prior research on admission prediction has primarily focused on narrow triage features and discrimination metrics, often overlooking operational factors, model calibration, and explainability. Early identification of patients likely to require admission could support proactive resource allocation and help reduce wait times.

Objective:

To develop and evaluate machine learning models for predicting hospital admission at the end of triage using routinely collected structured data, while addressing prior limitations regarding feature selection, model calibration, and explainability.

Methods:

We conducted a retrospective cohort study of adult ED visits between January 2015 and December 2022 at two academic hospitals within the McGill University Health Centre. We excluded visits with dispositions other than admission or discharge to ensure a binary outcome. Data were split into training (70%), validation (10%), and testing (20%) sets using stratified random sampling. We trained classifiers, including logistic regression, decision trees, random forests, artificial neural networks, and gradient boosting algorithms (XGBoost, LightGBM, and CatBoost), using demographics, vital signs, chief complaints, comorbidities, and historical utilization available at triage. We conducted sensitivity analyses on exclusion criteria and subgroup analyses across demographics and sites. Performance was evaluated using AUC-ROC, sensitivity, specificity, and calibration. We also performed a decision threshold analysis to assess operational flexibility and used SHAP (Shapley Additive Explanations) values to interpret feature contributions.

Results:

The final cohort included 526,145 visits (mean age 50.9 years, SD 21.1; 51.8% female), with 91,363 (17.4%) admitted. The XGBoost model achieved the highest performance with an AUC-ROC of 0.86 (95% CI 0.858-0.864), sensitivity of 0.67, and specificity of 0.86, outperforming the logistic regression baseline (AUC 0.84). Key predictors identified by SHAP values included age, Canadian Triage and Acuity Scale (CTAS) scores, arrival mode, and historical hospitalization frequency. Subgroup analyses confirmed robust performance across age, sex, and sites. Sensitivity analyses incorporating originally excluded dispositions into the admission category resulted in a slight performance improvement, confirming model stability. Operational analysis showed that adjusting the decision threshold to the 0.60–0.65 range improved the positive predictive value without introducing bias against older or higher-acuity patients.

Conclusions:

Hospital admission can be accurately predicted using only structured data available at triage. By incorporating broad feature sets and operational variables, machine learning models like XGBoost offer a feasible, explainable tool for early decision-making. These findings suggest that deploying such tools could enhance coordination between emergency and inpatient services to help alleviate overcrowding. Future work should focus on prospective validation and integration into real-time clinical workflows. Clinical Trial: N/A