JMIR Publications

ABSTRACT

Background:

Real-World Evidence (RWE) is increasingly used for clinical and regulatory decisions. However, the quality of real-world data (RWD) (e.g., electronic health records, claims, registries) remains a challenge to credible evidence generation.

Objective:

The objective of this Scoping Review is to synthesize the current literature on Real-World Evidence (RWE) Data Quality (DQ), specifically focusing on identifying and mapping the scope and nature of terminological inconsistency in DQ dimensions and assessment methods, and to analyze how these challenges impact the operationalization of the 'Fit-for-Purpose' (FFP) principle in RWE data assessment.

Methods:

This scoping review involved a systematic search of three databases (PubMed, Web of Science, and Scopus) for publications dated between January 2013 and July 2, 2025. The search targeted documents addressing frameworks, dimensions, and assessment methods related to data quality for RWE. To ensure comprehensive coverage, the primary search was supplemented using backward citation tracking (snowballing) to identify foundational literature. Data extraction focused on quality dimensions, their context of use, and their practical application.

Results:

Ninety-one eligible documents were included, concentrated in North America and Europe. Hybrid publications (academia-industry-government collaborations) constituted the majority (54.9%). A high prevalence (43%) of reported financial conflicts of interest (COI) was identified. The most cited RWD sources were EHRs, claims, and registries. Despite lacking a standardized lexicon, Completeness was the most reported Data Quality Dimension (DQD) (78 documents), followed by Accuracy (66), Consistency (47), Compliance (43), and Reliability (42). Validity and Currentness/Timeliness (41 each), Plausibility (39), and Relevance (37) were also prominent. "Fit For Purpose" (FFP) was central (57% of documents), defined as a context-dependent judgment of data suitability for a specific secondary application. While 41% of publications cited no specific framework, the Kahn et al. harmonized framework and the U.S. FDA’s RWE Program Framework were the most prominent.

Conclusions:

This review characterizes the RWE field as a collaborative yet fragmented ecosystem, concentrated in North America/Europe and marked by prevalent financial COIs. RWE credibility is fundamentally undermined by "terminological chaos": the lack of a standardized data quality lexicon. Key dimensions are used without consensus definitions, hindering reproducibility. This ambiguity makes the field's core FFP principle operationally unstable, as its components (e.g., reliability, relevance) also lack consensus. This subjectivity is high-risk given the prevalent COIs. A viable path forward requires moving beyond isolated frameworks toward an integrated, multi-stage solution: adopting foundational standards (e.g., ISO/IEC 25012, DAMA-DMBOK) for Upstream integrity, using Common Data Models (e.g., OMOP) for Midstream interoperability, and mandating Downstream transparency through protocol pre-registration and FAIR-aligned quality reports. Clinical Trial: The study protocol was registered with the Open Science Framework-OSF database under DOI 10.17605/OSF.IO/V5M86