JMIR Publications

ABSTRACT

Background:

Digital health interventions (DHIs) have the potential to enable public end-users, such as citizens and patients, to manage and improve their health. Although the number of available DHIs is increasing, examples of successfully established DHIs in public health systems are limited. To counteract a non-use of DHIs they should be comprehensively evaluated while integrating end-users. Unfortunately, there is a wide variability and heterogeneity according to the ways of evaluation which creates a methodological challenge.

Objective:

The scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview.

Methods:

This review was conducted based on the methodology framework for scoping reviews of Arksey & O’Malley and complies with the PRISMA-ScR guideline. Three scientific databases (Pubmed, Scopus, Science Direct) and grey literature were searched in April 2023. English and German studies between 2008 and 2023 were considered when a DHI that explicitly addresses public end-users was evaluated.

Results:

The search strategy identified 9618 publications, of which 160 were included. Of these finally included, 200 evaluations were derived and analysed. It is discovered that there is neither a consensus on methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators which results in a broad variety of evaluation practices. This aligns with observations of existing literature. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a result matrix is created where the findings were combined per DHI cluster. Additionally, general recommendations for evaluators of DHIs are formulated.

Conclusions:

The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the ways DHIs for public end-users are evaluated. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.