JMIR Publications

ABSTRACT

Background:

Delirium is an acute neurocognitive disorder which affects up to half of older hospitalized medical patients and can lead to dementia, longer hospital stays, increased health costs, and death. While delirium can be prevented and treated, it is difficult to identify and predict.

Objective:

We aim at developing and validating machine learning models by using Natural Language Processing (NLP) method to identify delirium in hospitalized admissions.

Methods:

Using data from GEMINI (https://www.geminimedicine.ca/), a Canadian hospital data and analytics network, a detailed manual review of medical records was conducted from nearly 4000 admissions at six Toronto-area hospitals, of which approximately 25% were labeled as having delirium. Using the data set collected from this study, we developed machine learning models with, and without, the benefit of NLP methods applied to diagnostic imaging reports, and asked the question “can NLP improve machine learning identification of delirium?”

Results:

Among the eligible 3,862 hospital admissions, 994 (25.74%) admissions were labeled as having delirium. Prediction and calibration of the models were satisfactory. The accuracy and ROC-AUC of the main model with NLP in the independent testing dataset were 0.807, and 0.930, respectively. While the results of accuracy and ROC-AUC of the main model without NLP in the independent testing dataset were 0.811, and 0.869, respectively. Model prediction was also found to be stable over the five year time period used in the experiment) with prediction for a prospective holdout test set being no worse than prediction for retrospective holdout test sets.

Conclusions:

Our machine learning model that included NLP (i.e., sentiment analysis in medical image description text mining) produced valid predictions of delirium identification.