JMIR Publications

ABSTRACT

Background:

Several prognostic scores have been proposed to predict functional outcome after an acute ischemic stroke (AIS). Most of them used structured information with the logistic regression method to develop prediction models. With the increased use of electronic health records and the progress in computational power, data-driven predictive modeling using machine learning (ML) techniques are gaining popularity in clinical decision-making.

Objective:

To investigate whether ML models created using unstructured text could improve the prediction of functional outcome at an early stage after AIS.

Methods:

We identified all consecutive patients hospitalized for the first time for AIS from October 2007 to December 2019 using a hospital stroke registry. The study population was randomly split into a training (n = 2885) and a test set (n = 962). Free text in the history of present illness and computed tomography reports was transformed into input variables using natural language processing. Models were trained using the extreme gradient boosting technique to predict a poor functional outcome at 90 days post-stroke. Model performance on the test set was evaluated using the area under the receiver operating characteristic curve (AUC).

Results:

The AUCs of text-only models ranged from 0.768 to 0.807, which were comparable to the National Institutes of Health Stroke Scale (NIHSS) score (0.811). Models using both patient age and text achieved AUCs of 0.823 and 0.825, which were similar to the model containing age and NIHSS (0.841), the preadmission comorbidities, level of consciousness, age, and neurological deficit (PLAN) score (0.837), and the Acute Stroke Registry and Analysis of Lausanne (ASTRAL) score (0.840). Adding variables from clinical text improved the predictive performance of each of the models containing age and NIHSS, PLAN, and ASTRAL scores (AUC increase from 0.841 to 0.861, 0.837 to 0.856, and 0.840 to 0.860, respectively).

Conclusions:

Unstructured clinical text can be utilized to improve the performance of existing models for predicting post-stroke functional outcome. However, considering the different terminologies used across health systems, each individual health system may consider applying the proposed methods to develop and validate its own models.