JMIR Publications

ABSTRACT

Background:

Adverse drug reactions (ADRs) are a significant public health issue. With the development of new drugs, drug safety, particularly ADRs, has become a more prominent concern [1]. The timely and efficient detection of ADRs from patientgenerated data is a critical task for public health monitoring. Every medication has potential benefits, but it does not always produce the desired effect for all users. Due to the limited scale and time of clinical trials, a comprehensive evaluation of a specific drug's outcomes is often challenging before it is released to the market.

Objective:

Adverse Drug Reactions (ADRs) remain a major public health concern, and traditional reporting systems often fail to detect them promptly. With the growing influence of Chinese social media platforms such as Weibo, Zhihu, and Xiaohongshu, usergenerated content provides a new opportunity for early ADR identification.This study applies sentiment analysis and machine learning techniques to detect potential ADRs from Chinese social media data. We collected and labeled 75,000 posts from the three platforms, distinguishing ADR-related from non-ADR content. After preprocessing and feature extraction using the TF-IDF method, nine machine learning algorithms were tested, including Support Vector Classification (SVC), Logistic Regression (LR), Random Forest (RF), Decision Tree (DT), XGBoost (XGB), and ensemble models.Among these, the XGBoost model achieved the best performance, with F1scores of 91.4% on Weibo, 94.3% on Zhihu, and 93.0% on Xiaohongshu. The results show that machine learning, particularly ensemble models, can effectively detect potential ADRs from unstructured social media text.This research demonstrates that sentiment analysis of Chinese social media data can complement traditional pharmacovigilance systems, enabling faster and more accurate identification of drug safety issues

Methods:

Data ingestion was executed via platform-specific Application Programming Interfaces (APIs) and advanced scraping scripts over a strict temporal horizon spanning from January 2024 to December 2024. Posts were captured utilizing a curated lexicon of 150 high-frequency target keywords, including colloquial Chinese terms for common drug categories, brand names, and vernacular expressions of somatic discomfort (e.g., '头疼 ' [headache], '恶心' [nausea]). No focus groups were utilized; the data acquisition strictly represents passive, non-interventional monitoring of public timelines.

Results:

This section presents the results of our empirical evaluation of nine machine learning algorithms for Adverse Drug Reaction (ADR) detection on Chinese social media. The performance of each classifier was measured using several key metrics: accuracy, precision, recall, and F1-score. The results for the Weibo, Zhihu, and Xiaohongshu datasets are summarized in Table 2. Additionally, the performance of each classifier is visualized in Figure 2 using the ROC AUC metric, which provides a comprehensive view of their ability to distinguish between ADR and non-ADR posts.

Conclusions:

Adverse Drug Reactions (ADRs) are negative side effects that can result from medication use. It is crucial to detect and analyze them to minimize clinical risks and reduce healthcare costs. This is particularly important for patient safety, as efficient reporting mechanisms can lead to timely detection. This paper evaluated the effectiveness of various machine learning algorithms in detecting ADRs on Chinese social media. We used a comprehensive data preprocessing pipeline, which included cleaning unstructured text and extracting features. To overcome the challenges of imbalanced datasets and prevent overfitting, we used both oversampling and undersampling techniques. This approach helped improve the overall performance and reliability of our models. The results consistently showed the superiority of the XGBoost (XGB) classifier. It demonstrated remarkable performance across the diverse datasets (Weibo, Zhihu, Xiaohongshu), achieving impressive accuracy rates of 92.5%, 94.8%, and 93.5%, respectively. These findings highlight the importance of choosing the right algorithms and showcase the immense potential of machine learning for complex tasks. For future work, our goal is to move beyond simply identifying sentences with ADRs to a more comprehensive approach that extracts specific ADR mentions from social media posts. This will require more advanced Natural Language Processing (NLP) techniques, including the use of large language models (LLMs) to extract semantic features. This could provide more accurate and detailed insights into ADR patterns, further enhancing our ability to proactively identify potential risks.