JMIR Publications

ABSTRACT

Background:

Opioid use disorder (OUD) remains a critical public health crisis in the United States. Despite widespread policy and clinical interventions, early identification of individuals at risk for developing OUD remains challenging due to limitations in traditional screening approaches and lack of individualized risk stratification methods. Machine learning (ML) methods offer an opportunity to develop timely, high-performing, and explainable predictive models that can enhance OUD prevention strategies in clinical settings.

Objective:

To develop and validate a ML model using electronic health record (EHR) data to predict 3-month risk of incident opioid use disorder (OUD) among adults initiating opioid therapy, and to stratify patients into clinically actionable risk groups.

Methods:

This prognostic modeling study used 2017–2022 OneFlorida+ EHR data to develop and validate ML algorithms predicting 3-month incident OUD risk. We included 182,083 adults (≥18 years) without cancer or hospice care who received ≥1 outpatient, non-injectable opioid prescription. Using 183 predictors measured in sequential 3-month intervals, we developed gradient boosting machine (GBM), elastic net (EN), least absolute shrinkage and selection operator (LASSO), and random forest (RF) models on randomly split training, testing, and validation sets. Model performance was assessed using C-statistics, predictive values, and NNE, with patients stratified into risk deciles for clinical applicability. Model explainability was assessed using SHapley Additive exPlanations (SHAP), and racial fairness was evaluated using Aequitas metrics.

Results:

In the validation sample (n=60,694), GBM (C-statistics=0.879, 95% CI=0.874-0.884) and EN (C-statistics=0.872, 95% CI=0.867-0.877) outperformed LASSO (C-statistics=0.846, 95% CI=0.840-0.851) and RF (C-statistics=0.798, 95%CI=0.792-0.804), with GBM model requiring the fewest predictors (n=75) for predicting 3-month incident OUD. Using the GBM algorithm to predict subsequent 3-month OUD risk, the top decile subgroup had a PPV of 15.06%, NPV of 99.8%, and NNE of 89. The top decile (n=6696) captured ~68% of patients with OUD. SHAP analysis identified age, number of outpatient visits, history of back and other pain conditions, comorbidity burden, and opioid prescribing patterns as the strongest predictors of incident OUD. Fairness assessment showed acceptable false negative rate parity across racial groups.

Conclusions:

Building on previous claims-based ML models to predict OUD in the Medicare population, our study demonstrated high discriminative performance and effective risk stratification for incident OUD using OneFlorida+ EHR data, with potential to inform early intervention. ML algorithms leveraging real-world EHR data can support clinical decision-making to proactively identify patients at risk for OUD. Clinical Trial: Not applicable