JMIR Publications

ABSTRACT

Background:

Despite a growing number of systematic reviews on digital health interventions (DHIs), many do not sufficiently support the recognition of conclusive evidence. Methodological shortcomings may impede the identification and communication of robust findings.

Objective:

This study examines to what extent systematic reviews apply methodological standards - particularly the specification of PICO elements (Population/Problem, Intervention, Comparison, Outcome) - and how this relates to the likelihood of conclusive evidence recognition. It focuses on the assessment at abstract-level, as abstracts are used independently of the specific review choice to screen and select studies for evidence synthesis.

Methods:

Following PRISMA guidelines, we conducted a comprehensive database search (2011–2023). From 2,528 eligible systematic reviews, a random sample of 250 abstracts was analyzed descriptively. Abstracts were assessed for PICO specification and evidence conclusiveness in the context of further study characteristics.

Results:

48% of reviews showed low or very low PICO specification, and 64% reported inconclusive or weak evidence. Higher specification of outcomes and problems was moderately associated with conclusive evidence. Beside the formulation of the research question along the PICO scheme, the search and screening strategies, e.g., RCT focus, use of appropriate search terms, search periods, databases, and appraisal tools, were also found to hinder evidence recognition.

Conclusions:

Limited application of methodological standards restricts evidence recognition in DHI reviews. A structured PICO-based framework which is aligned with current research, builds on well-established categories, and provides clear and differentiated definitions may enhance the focus, and evidentiary strength of future reviews.