Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Aug 12, 2020
Open Peer Review Period: Aug 12, 2020 - Aug 18, 2020
Date Accepted: Jun 7, 2021
(closed for review but you can still tweet)
Machine learning-based PRediction of ICU DElirium (PRIDE) Algorithm for Delirium Prediction in Intensive Care Units: A Retrospective Study
ABSTRACT
Background:
Delirium occurs frequently among patients admitted to intensive care unit (ICU). There is only limited evidence to support interventions to treat or resolve delirium in patients who have already developed delirium. Therefore, the early recognition and prevention of delirium is important in the management of critically ill patients.
Objective:
This study aimed to develop and validate the PRIDE (PRediction of ICU DElirium) model with machine learning using electronic health record data for delirium prediction within 24 hours from ICU admission.
Methods:
This is a retrospective cohort study performed at a tertiary referral hospital with 120 ICU beds. Machine learning-based PRIDE (PRediction of ICU DElirium) models were developed using patient data from the first 2 years of the study period and validated using patient data from the last 6 months. eXtreme Gradient Boosting (XGBoost), random forest (RF), deep neural network (DNN), and logistic regression (LR) were used. The PRIDE model was externally validated using MIMIC-III data.
Results:
We only included patients who were 18 years or older at the time of admission and who stayed in the medical or surgical ICU. A total of 37,543 cases were collected. After patient exclusion, 12,409 remained as our study population, of which 3,816 (30.8%) patients experienced delirium incidents during the study period. The MIMIC-3 dataset, based on the exclusion criteria, out of the 96,016 ICU admission cases, 2,061 cases were included, and 272 (13.2%) delirium incidents occurred. In the internal validation, the area under the receiver operating characteristics (AUROC) for XGBoost, RF, DNN, and LR was 0.919 (95% CI 0.919–0.919), 0.916 (95% CI 0.916–0.916), 0.881 (95% CI 0.878–0.884), and 0.875 (95% CI 0.875–0.875), respectively. Regarding the external validation, the best AUROC was 0.721 (95% CI 0.72–0.721), 0.697 (95% CI 0.695–0.699), 0.655 (95% CI 0.654–0.657), and 0.631 (95% CI 0.631–0.631) for RF, XGBoost, DNN, and LR, respectively. The Brier score of the XGBoost model is 0.094, indicating that it is well calibrated.
Conclusions:
A machine learning approach based on electronic health record data can be used to predict delirium within 24 hours of ICU admission. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.