JMIR Publications

ABSTRACT

Background:

Overdose and suicide deaths due to non-prescribed fentanyl have increased significantly. Fortunately, there are treatments available that could reduce the risk of death. However, for healthcare systems to implement programs addressing the needs of patients who use non-prescribed fentanyl, they must first be able to identify them. International Classification of Diseases (ICD) codes are not a good option for identifying patients who use non-prescribed fentanyl for two reasons. First, opioid use disorder (OUD) diagnoses are not always coded. Second, when they are coded, ICD codes for OUD do not specify whether fentanyl is being used rather than other, less lethal opioids.

Objective:

To develop natural language processing (NLP) approaches to identifying current non-prescribed fentanyl use in electronic health record (EHR) documentation.

Methods:

This retrospective cross-sectional study included Veterans Health Administration (VHA) patients seen in the VHA between 4/5/23 and 12/23/24. A term list was developed to identify fentanyl-related mentions in clinical text, and 250-character snippets surrounding identified mentions were extracted. Veterans (n=3,878) were randomly sampled from five predefined groups based on the presence of one of four terms (fentanyl, fent, blues, tranq) in their EHR documentation. Physician annotators classified snippets into “non-prescribed fentanyl use,” “prescribed fentanyl use,” or “other.” Labeled data was used to train, validate, and compare the ability of penalized logistic regression, Bio-Clinical BERT, LLaMA 3 8B, and Mistral 7B to classify snippets. Model performance was evaluated using weighted average precision, recall, and F1 scores.

Results:

Results:

Among 7,389 snippets, 9.6% were “non-prescribed fentanyl use,” 40.3% were “prescribed fentanyl use,” and 50% were “other.” LLaMA 3 8B achieved the highest weighted average F1 score (0.954) outperformed penalized logistic regression with a small statistically significant difference, while differences between Mistral 7B and LLaMA 3 8B were not statistically significant. Top discriminative terms included “consumption,” “purchased,” and “streets” for non prescribed fentanyl use; “dressing,” “patch(es),” and “1500” for prescribed; and “laced,” “testing,” and “fears” for other fentanyl use.

Conclusions:

NLP can accurately identify non-prescribed fentanyl use in EHR documentation. This approach may support risk prediction and targeting of interventions to patients exposed to non-prescribed fentanyl.