JMIR Publications

ABSTRACT

Background:

Although there is a growing interest in prediction models based on electronic medical record (EMR), to identify patients at risk of adverse cardiac events following invasive coronary treatment, robust models fully utilizing EMR data are limited.

Objective:

We aimed to develop and validate machine-learning (ML) models using diverse fields of EMR to predict risk of 30-day adverse cardiac events after percutaneous intervention or bypass surgery.

Methods:

EMR data of 5,184,565 records of 16,793 patients at a quaternary hospital between 2006-2016, was categorized into static basic (e.g. demographics), dynamic time-series (e.g. laboratory values), and cardiac-specific data (e.g. coronary angiography). The data were randomly split into training, tuning, and testing sets in a ratio of 3:1:1. Each model was evaluated with 5-fold cross-validation and with an external EMR-based cohort at a tertiary hospital. Logistic regression (LR), random forest (RF), gradient boosting machine (GBM), and feedforward neural network (FNN) algorithms were applied. Primary outcome was 30-day mortality following invasive treatment.

Results:

GBM showed the best performance with area under the receiver operating characteristic curve (AUROC) of 0.99; RF had a similar AUROC of 0.98. AUROCs of FNN and LR were 0.96 and 0.93, respectively. GBM had the highest area under the precision-recall curve (AUPRC) of 0.80 and those of RF, LR and FNN were 0.73, 0.68, and 0.63, respectively. All models showed low Brier scores of <0.1 as well as highly fitted calibration plots, indicating a good fit of the ML-based models. On external validation, the GBM model demonstrated maximal performance with AUROCs 0.90, while FNN had AUROC of 0.85. The AUROC of LR and RF were slightly lower at 0.80, and 0.79, respectively. The AUPRCs of GBM, LR, and FNN were similar at 0.47, 0.43, and 0.41, respectively, while that of RF was lower at 0.33. All models showed low Brier scores of 0.1. Among the categories in the GBM model, time-series dynamic data demonstrated high AUROC of >0.95, contributing majorly to the excellent result

Conclusions:

Exploiting diverse fields of EMR dataset, the ML-based 30-days adverse cardiac event prediction models performed outstanding, and the applied framework could be generalized for various healthcare prediction models.ts.