JMIR Publications

ABSTRACT

Background:

Patient-generated drug reviews are increasingly used as complementary sources of drug safety information, and previous research has explored their application in pharmacovigilance. However, it remains unclear whether the frequencies of adverse events (AEs) reported in reviews systematically differ from regulatory label frequencies and, if so, whether such differences vary by AE type, especially between experiential symptoms and externally verifiable signs. If such distortions exist, making certain adverse events appear more common or less common than they are, they could lead to inaccurate risk perceptions and medication decisions.

Objective:

We examined whether AEs in patient-generated drug reviews are overreported relative to regulatory benchmarks and whether these patterns vary between two types of AEs: symptoms and signs. We further examined whether these reporting patterns vary with clinical seriousness and post market information maturity, and whether additional heterogeneity emerges across Patient-Reported Outcomes Measurement Information System (PROMIS) domains for symptoms and types of objective verifiability for signs.

Methods:

We randomly selected 100 drugs covered by WebMD and collected their available reviews posted between 2007 and 2025. We compared AE frequencies in reviews with corresponding drug label frequencies obtained from the Side Effect Resource (SIDER) 4.1 database. AboveBenchmark, a binary indicator of overreporting, indicated whether the review-based adverse-event frequency exceeded the corresponding SIDER label frequency. AEs were classified as symptoms or signs and further categorized by seriousness, PROMIS health domains, and objective verifiability (externally observable or clinically testable) using a large language model (GPT-4o). We used logistic regression to examine overreporting, with drug-level and adverse event–level controls and indicators for Anatomical Therapeutic Chemical (ATC) drug class and Medical Dictionary for Regulatory Activities (MedDRA) system organ class (SOC) category.

Results:

Matching yielded 2524 drug–AE pairs across 76 drugs from 29,720 reviews; 2520 pairs remained after excluding pairs with equal review and SIDER benchmark frequencies. Symptom-type AEs were less likely to be overreported relative to regulatory benchmarks (β = −0.471, 95% CI −0.744 to −0.198; P<.001) while sign-type AEs were more likely (β = 0.546, 95% CI 0.338 to 0.755; P<.001) to be overreported. Adverse event seriousness reduced the underreporting tendency of symptom-type adverse events, while older drug age was associated with a stronger overreporting tendency for sign-type adverse events. Additional heterogeneity analyses showed distinct patterns across PROMIS domains for symptoms and across types of objective verifiability for signs. Findings were robust to alternative outcome definitions and benchmark specifications.

Conclusions:

Symptom-type AEs were less likely to be overreported, whereas sign-type AEs were more likely to be overreported, with reporting patterns differing by clinical seriousness, drug age, and type-specific heterogeneity. Distinguishing between symptoms and signs provides a practical framework for reducing systematic bias in downstream modeling, machine learning, and AI-mediated health information systems.