JMIR Publications

ABSTRACT

Background:

Optimal glycemic control is essential for limiting diabetes–related complications, but the risk of hypoglycemia constrains treatment intensification. Current hypoglycemia surveillance methods relying on diagnosis codes substantially underreport events, and improved methods for identifying hypoglycemia from electronic health records (EHRs) are needed.

Objective:

To develop and validate an artificial intelligence (AI) model that can use EHR clinical notes to retrospectively identify severe hypoglycemia events among outpatients with type 2 diabetes (T2D) presenting to the Emergency Department (ED) or requiring an Inpatient Admission (IP).

Methods:

Using Veterans Health Administration data, we identified a sample of T2D patients in 2019 and classified hypoglycemia via ICD-10 codes or outpatient glucose <70 mg/dL (Hypo+) versus controls (Hypo−). Text corpora included clinical notes near ED encounters (±24 hours) or admission (−12 to +24 hours). We created a hybrid classifier to identify hypoglycemic patients by combining rule-based feature extraction with random forest–based machine learning. A rule–based concept extraction module within the RapTAT natural language processing system generated 44 assertion-aware features. Random forest models were developed and trained using these features and leave-one-out cross-validation and then tested on held-out sets. We created one model for hypoglycemia during ED visits and a second for IP admissions with hypoglycemia. Feature importance was evaluated across both models, and performance was evaluated across different cohorts (2018 vs. 2019) and clinical contexts (presence of low blood glucose <70 mg/dL, hypoglycemia treatment).

Results:

When evaluated across all clinical contexts, the ED model achieved a normalized positive predictive value (PPVN) of 82%-88% and sensitivity of 88%-92%, with specificity of 88% (2018) and 80% (2019). Performance was highest in the context of documented low glucose and hypoglycemia treatment. The IP model performed better overall: PPVN 89%–94%, sensitivity 85%-93%, and specificity 93%-94%. Requiring ≥5 documents improved IP accuracy. Feature importance rankings correlated across models (r=0.74); top predictors included counts and ratios of low glucose phrases and document–level coverage of hypoglycemia concepts. Hypoglycemia cases skewed older, male, and had higher representation of African American patients.

Conclusions:

AI in the form of a hybrid classification model applied to unstructured EHR notes accurately identified severe hypoglycemia in outpatient T2D populations, outperforming historically reported diagnosis code–based sensitivity and specificity. Models were most effective for admission–related hypoglycemia and when corroborating glucose values and treatment were present. These tools enable scalable surveillance, that may inform safer diabetes management, and support quality improvement.