Previously submitted to: Journal of Medical Internet Research (no longer under consideration since Dec 28, 2024)
Date Submitted: Sep 11, 2024
Open Peer Review Period: Sep 12, 2024 - Nov 7, 2024
(closed for review but you can still tweet)
NOTE: This is an unreviewed Preprint
Warning: This is a unreviewed preprint (What is a preprint?). Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn (a note "no longer under consideration" will appear above).
Peer review me: Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period (in this case, a "Peer Review Me" button to sign up as reviewer is displayed above). All preprints currently open for review are listed here. Outside of the formal open peer-review period we encourage you to tweet about the preprint.
Citation: Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author).
Final version: If our system detects a final peer-reviewed "version of record" (VoR) published in any journal, a link to that VoR will appear below. Readers are then encourage to cite the VoR instead of this preprint.
Settings: If you are the author, you can login and change the preprint display settings, but the preprint URL/DOI is supposed to be stable and citable, so it should not be removed once posted.
Submit: To post your own preprint, simply submit to any JMIR journal, and choose the appropriate settings to expose your submitted version as preprint.
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Interpretable prediction of hospital mortality in bleeding critically ill patients based on machine learning and SHAP
ABSTRACT
Background:
Hemorrhage is a prevalent and critical condition in the intensive care unit (ICU), marked by high incidence, elevated mortality rates, and substantial therapeutic challenges. Accurate prediction of mortality in patients with hemorrhage is essential for the development of personalized prevention and treatment strategies. Nevertheless, the implementation of effective predictive models in clinical practice remains limited, largely due to the current gap in robust and interpretable prediction tools.
Objective:
This study aimed to develop an interpretable model for predicting mortality risk in critically ill patients with hemorrhage in intensive care units (ICUs). The SHapley Additive exPlanation (SHAP) method was applied to interpret the extreme gradient boosting (XGBoost) model, allowing for the exploration of key prognostic factors in this patient population.
Methods:
In this retrospective cohort study, we developed and evaluated the performance of a predictive model using data from the eICU Collaborative Research Database (eICU-CRD). Data from the first 24 hours of each ICU admission were extracted, with the dataset randomly split into a training set (80%) and a validation set (20%). The predictive performance of the XGBoost model was compared to four other machine learning models using the area under the curve (AUC) as the metric. The SHapley Additive exPlanation (SHAP) method was employed to interpret the XGBoost model. Following initial validation, external validation was performed using data from a Chinese retrospective cohort, Refrain, which focuses on hemorrhage and coagulopathy in critically ill patients.
Results:
A total of 10306 eligible patients with hemorrhage were included in the final cohort for this study. The observed in-hospital mortality of patients with hemorrhage was 11.5%. Comparatively, the XGBoost model had the highest predictive performance among five models with an area under the curve (AUC=0.81) , whereas LR had the poorest generalization ability (AUC=0.726). The decision curve showed that the net benefit of the XGBoost model surpassed those of other machine learning models at 10%~30% threshold probabilities. The SHAP method reveals the top 15 predictors of hemorrhage according to the importance ranking, and the bilirubin level was recognized as the most important predictor variable. Additionally, in the external validation using the REFRAIN cohort, the XGBoost model demonstrated robust predictive performance with an AUC of 0.776.
Conclusions:
The interpretable predictive model enhances the accuracy of mortality risk prediction in ICU patients with hemorrhage, enabling clinicians to devise more effective treatment plans and optimize resource allocation. Moreover, the interpretability framework increases model transparency, thereby facilitating clinicians' understanding and trust in the reliability of the predictive model.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.