JMIR Publications

ABSTRACT

Background:

Diagnostic errors are an underappreciated cause of preventable mortality in hospitals and pose risk for severe patient harm and increase hospital length of stay.

Objective:

We aimed to explore the potential of machine learning (ML) and natural language processing (NLP) techniques in improving diagnostic safety surveillance. We conducted a rigorous evaluation of the feasibility and potential to use electronic health records clinical notes and existing case review data.

Methods:

Case review data from one large health system comprised of ten hospitals in the mid-Atlantic region of the U.S. from February 2016 to September 2021 was analyzed. The case review outcome included opportunities for improvement (OFIs), including diagnostic OFIs. To supplement case review data, EHR clinical notes were extracted and analyzed. Simple logistic regression model along with three forms of logistic regression models (i.e., LASSO, Ridge, and Elastic net) with regularization functions were trained on this data to compare classification performances in classifying patients who experienced diagnostic errors during hospitalization. Further, statistical tests were conducted to find significant differences between female and male patients who experienced diagnostic error.

Results:

In total, 126 patients (out of 1,704, 7.4%) had been identified by case reviewers as having experienced at least one diagnostic error. Patients who had experienced diagnostic error were grouped by sex: 59 of the 830 females (7.1%) and 67 of the 874 males (7.7%). Among the patients who experienced diagnostic error, female patients were older (median = 9 years, p = 0.02), had higher rates of being admitting through general/internal medicine (69.5% versus 47.8%, p = 0.01), lower rates of cardiovascular-related admitted diagnosis (11.9% versus 28.4%, p = 0.02), and lower rates of being admitting through neurology department (2.3% versus 13.4%, p = 0.04). The Ridge model achieved the highest area under the receiver operating characteristic curve (0.885), specificity (0.797), positive predictive value (0.24), and F-1 score (0.369) in classifying patients who were at higher risk of diagnostic errors among hospitalized patients.

Conclusions:

Our findings demonstrate that NLP can be a potential solution to more effectively identifying and selecting potential diagnostic error cases for review and therefore, reducing case review burden.