JMIR Publications

ABSTRACT

Background:

Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection and remains one of the leading causes of mortality in intensive care units (ICUs) worldwide. Although high-performance “black-box” machine learning (ML) models have shown promise for early sepsis detection, their lack of transparency limits clinician trust and adoption in real-world clinical decision support.

Objective:

This study aimed to develop and comparatively evaluate ML models for early sepsis prediction using ICU time-series data and to integrate explainable artificial intelligence (XAI) techniques—Shapley additive explanations (SHAP) and local interpretable model-agnostic explanations (LIME)—to provide transparent, clinically meaningful, and patient-level explanations of model predictions.

Methods:

We used the PhysioNet Challenge 2019 sepsis dataset, comprising hourly ICU patient records (546,123 records; 44 features), including vital signs, laboratory measurements, and demographic information. After preprocessing (removal of highly missing features, median imputation, and feature selection), 16 clinically relevant features were retained. Data were split into training (80%, n=436,898) and testing (20%, n=109,225) sets using stratified sampling. Four ML models—logistic regression, random forest, XGBoost, and LightGBM—were trained and compared using accuracy, precision, recall, F1-score, ROC-AUC, and PR-AUC. The final model was selected using a predefined emphasis on sensitivity (recall) for early-warning purposes. Multiple classification thresholds (0.50–0.90) were evaluated to examine the precision–recall trade-off. Global and local model interpretability were assessed using SHAP (summary and waterfall plots) and LIME, and their explanation consistency was compared at both the individual patient and overall feature levels.

Results:

LightGBM achieved the highest recall (63.80%) among the evaluated models at the default threshold (0.50), with an accuracy of 83.56%, precision of 8.12%, F1-score of 14.41%, ROC-AUC of 82.68%, and PR-AUC of 13.39%, and was therefore selected as the final model because the predefined model-selection criterion emphasized recall for the early-warning objective. Threshold analysis showed a clear precision–recall trade-off: increasing the threshold to 0.85 improved precision from 8.12% to 18.26% but reduced recall from 63.80% to 28.69%, yielding the highest F1-score (22.31%) among the thresholds tested. SHAP analysis identified ICU length of stay (ICULOS), heart rate, respiratory rate, age, and temperature as the most influential predictors of sepsis risk. Patient-level comparison between SHAP and LIME showed consistent identification of key contributing features (e.g., heart rate, respiratory rate, ICULOS, and gender) for individual predictions, with an overall SHAP–LIME feature-agreement ranking led by ICULOS, heart rate, and respiratory rate.

Conclusions:

We developed an interpretable ML framework for early sepsis detection that combines comparative model evaluation with SHAP- and LIME-based explainability. LightGBM, selected for its higher sensitivity to sepsis cases, together with consistent SHAP and LIME explanations, may support more transparent early-warning model interpretation. Further external validation and clinical evaluation are needed before real-world deployment.