JMIR Publications

ABSTRACT

Background:

Acute kidney injury (AKI) is commonly encountered in clinical practice and is associated with poor patient outcomes and increased healthcare costs. Despite it posing significant challenges for clinicians, effective measures for AKI prediction and prevention are lacking. Previously published AKI prediction models mostly have a simple design without external validation. Furthermore, little is known regarding the process of linking model output and clinical decisions due to the blackbox nature of neural network models.

Objective:

We aimed to present an externally validated recurrent neural network (RNN)-based continuous prediction model for in-hospital AKI and show applicable model interpretations in relation to clinical decision support.

Methods:

Study populations were all patients aged ≥ 18 years who were hospitalized for more than 48 h between 2013 and 2017 in two tertiary hospitals in Korea (Seoul National University Bundang Hospital and Seoul National University Hospital). All demographics, laboratory values, vital signs, and clinical conditions were obtained from the EHR of each hospital. We developed two-stage hierarchical prediction models (model 1 and model 2) using RNN algorithms. The outcome variable for Model 1 was the occurrence of AKI within 7 days from the present. Model 2 predicted the future trajectory of Cr values up to 72 h. The performance of each developed model was evaluated with the internal and external validation dataset. For the explainability of our models, different model-agnostic interpretation methods were used, including Shapley additive explanations, partial dependence plots, individual conditional expectation, and accumulated local effects plots.

Results:

We included 69,081 patients in the training, 7,675 in the internal validation, and 72,352 in the external validation cohorts for model development after excluding cases with missing data and those with an estimated glomerular filtration rate <15 ml/min/1.73m2 or end-stage kidney disease. Model 1 predicted any AKI development with an area under the curve (AUC) of 0.88 (internal validation) and 0.84 (external validation), and stage 2 or higher AKI development with an AUC of 0.93 (internal validation) and 0.90 (external validation). Model 2 predicted the future creatinine values within 3 days with mean square errors of 0.04-0.09 for patients with higher risks of AKI and 0.03-0.08 for those with lower risks. Based on the developed models, we showed AKI probability according to feature values in total patients and each individual with partial dependence, accumulated local effects, and individual conditional expectation plots. We also estimated the effects of feature modifications such as nephrotoxic drug discontinuation on future creatinine levels.

Conclusions:

We developed and externally validated a continuous AKI prediction model using RNN algorithms. Our model could provide real-time assessment of future AKI occurrences and individualized risk factors for AKI in general inpatient cohorts. These suggest approaches to support clinical decisions based on prediction models for in-hospital AKI.