JMIR Publications

ABSTRACT

Background:

Postoperative delirium (POD) is a common complication after major surgery and is associated with poor outcomes in older adults. Early identification of patients at high risk of POD can enable targeted prevention efforts. However, existing POD prediction models require inpatient data collected during the hospital stay, which delays predictions and limits scalability.

Objective:

To develop and externally validate a machine learning-based prediction model for postoperative delirium using routine electronic health record (EHR) data.

Methods:

We identified all surgical encounters from 2014-2021 for patients aged 50 years and older who underwent an operation requiring general anesthesia with a length of stay of at least one day at three Indiana hospitals. Patients with pre-existing dementia or mild cognitive impairment were excluded. POD was identified using Confusion Assessment Method records and delirium ICD codes. Controls without delirium or nurse-documented confusion were matched to cases by age, sex, race, and year of admission. We trained logistic regression, random forest, extreme gradient boosting (XGB), and neural network models to predict POD using 143 features derived from routine EHR data available at the time of hospital admission. Separate models were developed for each hospital using surveillance periods of 3 months, 6 months, and 1 year before admission. Model performance was evaluated using the area under the receiver operating curve (AUC). Each model was internally validated using holdout data and externally validated using data from the other two hospitals. Calibration was assessed using calibration curves.

Results:

The study cohort included 7,167 delirium cases and 7,167 matched controls. XGB outperformed all other classifiers. AUCs were highest for XGB models trained on 12 months of pre-admission data. The best performing XGB model achieved an AUC of 0.79 (SD 0.01) on the holdout set, which decreased to 0.69-0.74 when externally validated on data from other hospitals.

Conclusions:

Our routine EHR-based POD prediction models demonstrated good predictive ability using a limited set of pre-admission and surgical variables, though their generalizability varied. The proposed models could be used as a scalable, automated screening tool to identify patients at high risk of POD at the time of hospital admission.