Previously submitted to: Journal of Medical Internet Research (no longer under consideration since Dec 22, 2023)
Date Submitted: Oct 19, 2023
Open Peer Review Period: Oct 19, 2023 - Dec 14, 2023
(closed for review but you can still tweet)
NOTE: This is an unreviewed Preprint
Warning: This is a unreviewed preprint (What is a preprint?). Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn (a note "no longer under consideration" will appear above).
Peer review me: Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period (in this case, a "Peer Review Me" button to sign up as reviewer is displayed above). All preprints currently open for review are listed here. Outside of the formal open peer-review period we encourage you to tweet about the preprint.
Citation: Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author).
Final version: If our system detects a final peer-reviewed "version of record" (VoR) published in any journal, a link to that VoR will appear below. Readers are then encourage to cite the VoR instead of this preprint.
Settings: If you are the author, you can login and change the preprint display settings, but the preprint URL/DOI is supposed to be stable and citable, so it should not be removed once posted.
Submit: To post your own preprint, simply submit to any JMIR journal, and choose the appropriate settings to expose your submitted version as preprint.
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Developing a Machine Learning Model to Predict In-hospital Mortality in Sepsis-complicated Acute Pancreatitis: A Retrospective Study Utilizing MIMIC-III, MIMIC-IV and eICU Databases
ABSTRACT
Background:
Acute pancreatitis, a potentially lethal ailment that sees a surge in mortality rates when sepsis onset occurs, is quite prevalent. Given its severity, early intervention and tailored treatment become pivotal--underscored by the urgent need to anticipate mortality in patients plagued by acute pancreatitis complicated by sepsis. This highlights the crucial demand for the development of a cogent, interpretable mortality prediction model.
Objective:
The objective of this study is to establish and validate a machine learning predictive model for determining the risk of in-hospital mortality in patients with acute pancreatitis (AP) complicated by sepsis. AP is a prevalent and potentially fatal inflammatory condition of the pancreas. When sepsis complicates AP, the likelihood of multiple organ dysfunction syndrome (MODS) increases, resulting in a very poor prognosis.
Methods:
This study utilized patient cohorts from the MIMIC-III, MIMIC-IV, and eICU databases to conduct a regression analysis for both model development and validation. Feature selection was performed using the LASSO method and feature importance based on SHAP values, resulting in the establishment of 11 different machine learning predictive models. A stacked ensemble learning predictive model was developed using the Stacking ensemble algorithm. Model performance was evaluated using various metrics, including areas under the receiver operating characteristic (ROC) curves (AUC), PR curves, accuracy, recall, and F1 scores. The models were explained using the Kernel-SHAP algorithm and LIME algorithm.
Results:
An ensemble learning predictive model named AQlearner was developed based on feature variables screened via SHAP importance. The AUC value of AQlearner not only exceeded that of other individual models but also was higher than any machine learning predictive model established on variables screened out via LASSO, with an AUC reaching up to 0.873. Other performance indicators also showed excellent results. In the external validation cohort, the model's performance remained outstanding and was eventually selected. The Kernel-SHAP algorithm and LIME algorithm were used to interpret the machine learning models, and the results showed that this approach is feasible.
Conclusions:
This study aimed to construct a machine-learning prediction model to evaluate the risk of in-hospital mortality in patients with acute pancreatitis complicated by sepsis. In clinical practice, feature selection based on the SHAP algorithm outperforms that based on Lasso. Furthermore, our ensemble learning predictive model exhibits superior performance compared to traditional single machine learning predictive models. The interpretability of machine learning models can be enhanced through the Kernel-SHAP algorithm and LIME algorithm. This SHAP feature importance-based model is highly suitable for clinical practice, as it can assist physicians in comprehending the underlying causes of predicted death and evaluating the in-hospital outcomes of critically ill patients.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.