JMIR Publications

ABSTRACT

Background:

: The opioid crisis poses a significant global health challenge in the U.S, with increasing overdoses and death rates due to opioids mixed with other illicit substances. Various strategies have been developed by federal and local governments and health organizations to address this crisis. One of the most significant objectives is to understand the epidemic through better health surveillance, and machine learning techniques can support this by identifying opioid overdose users through the analysis of social media data, as many individuals may avoid direct testing but still share their experiences online

Objective:

In this study, we take advantage of recent developments in machine learning that allow for insights into patterns of opioid use and potential risk factors in a less invasive manner using self-reported information available on social platforms.

Methods:

This study utilized YouTube comments collected between December 2020 and March 2024, in which individuals shared their self-reported experiences of opioid drugs mixed with other substances. We manually annotate our dataset into multi-class categories, capturing both the positive effects of opioid use, such as pain relief, euphoria, and relaxation, and negative experiences, including nausea, sadness, and respiratory depression, to provide a comprehensive understanding of the multifaceted impact of opioids. By analyzing this sentiment, we employed state-of-the-art four machine learning models, two deep learning models, three transformer models, and one large language model such as GPT-3.5 Turbo that predicts overdose risks to improve healthcare response and intervention strategies.

Results:

Our proposed methodology (GPT-3.5 Turbo) was highly precise and accurate, helping to automatically identify the sentiment based on the adverse effects of opioid drug combinations to high-risk drug use in YouTube comments. Our proposed methodology demonstrated the highest achievable F1-score of 0.77 and a 0.17% performance improvement over traditional machine learning models (XGB, 0.66).

Conclusions:

This study demonstrates the potential of leveraging machine learning and large language models, such as GPT-3.5 Turbo, to analyze public sentiment surrounding opioid use and its associated risks. By utilizing YouTube comments as a rich source of self-reported data, the study provides valuable insights into both the positive and negative effects of opioids, particularly when mixed with other substances. The proposed methodology significantly outperformed traditional models, contributing to more accurate predictions of overdose risks and enhancing healthcare responses to the opioid crisis.